What's my bandwidth?

[wuurp]

I've attached a screenshot of the MRTG graph on my server for yesterday. Could someone please estimate the amount of bandwidth used in that last 24 hours. Thank you. Extra credit for showing your work.


Oh, drat. I meant for this to go in VPS, Dedicated.

[Vin DSL]

Dude, I dunno!

Outgoing is practically '0'. Incoming is '1000 Mb/s' give or take...

SUP with that?!?!?

[wuurp]

Other way I think. Blue line in, green bars out, relative to the server.

And not sure where you're getting 1000Mb/s. Top of the graph is only 6.4Mb/s.

What I'm asking is how many GB transferred in those 24 hours.

[wmgcf]

how hard is it to get the table values instead of the chart?

[wuurp]

That's all I have access to in Nixcore.

Here's a hint. Using a trick with Imagemagick, the average Mb/s is a bit under 2, but since it didn't count incoming, let's just say the average over the 24 hours is 2Mb/s.

[wuurp]

No one?

Ok, here is what I get.

2Mb/s x 1 byte/8bits x 86400s/day = 21,600Mbytes/day or 21.6 GB.

Right?

So, if MRTG says I used 49 GB over that time period, would you suspect something is wrong?

[wmgcf]

No one?

Ok, here is what I get.

2Mb/s x 1 byte/8bits x 86400s/day = 21,600Mbytes/day or 21.6 GB.

Right?

So, if MRTG says I used 49 GB over that time period, would you suspect something is wrong?

sorry you can't get the table values. i wish you could just get them- cuz then you can auto track trends and have an alert at fixed or standard deviation thresholds.

[wuurp]

Thank you for the reply, but that's not the point.

The point is that the bandwidth that JAG says I am using is wrong, dractically wrong.

Ok, more MRTG graphs from last night, before and after midnight when the bandwidth usage is calculated. (see attached)

The top one is from before midnight. It shows my bandwidth usage at 1282.82 GB. Second one is just after the bandwidth was recalucated, and shows 1316.13 GB. The third one is from about 90 minutes later, and it now shows my bandwidth usage at 1331.83 GB.

So I would assume that means I have used 1331.83 - 1282.82 = 49.01 GB. That covers the time period on the graph between the red lines. Now, 49GB would be

(49GB x 1024MB/GB x 8bits/byte) / (24 hours * 60m/h * 60s/m)

= 4.65Mb/s average through the day

It looks to me like there is only one 5 min period that even reached that rate. It's certainly not the average rate. As I mentioned above, I estimated the average at about 2Mb/s.

I have tried to tell this to support, and I get non-answers like "the graph is updated every 5 minutes". I have been discussing this with support for 3 days now. Well, I write examples using their data why I don't believe the bandwidth usage is accurate, they ignore what I say.

I have also shown bandwidth usage as reported by vnstat. It reads the byte counter in /proc/net/dev. Is it 100% reliable? I doubt it, but it certainly gives me more accurate values than the MRTG graph is giving.

I asked for the ticket to be escalated to someone that would answer my question. 16 hours later I get the response "The graph is updated every five minutes. The usage calculations are only once a day around midnight. Please let us know if we can be of further assistance."

Needless to say, I am getting very frustrated with support.

And before anyone thinks I'm trying to cheat on bandwidth or something, this all started because I reported that the MRTG was vastly underreporting my bandwidth usage. It showed only 8.4 GB bandwidth for the first 20 days of the month, and I always use more than that per day. Seems like an error that under reports bandwidth usage gets fixed, but if it over reports, then it's not an error. I should have kept my mouth shut.

[Ron]

Maybe it's counting the whole node as your bandwidth; perhaps it's counting backup as bandwidth?

[JPC-Greg]

Its not using an average, never has been. If your having those big bursts above 10mb quite often your going to drag that bw rate up. We go by the 95th standard , though to be honest I had devs drop it to 80th percent years ago since bw was so cheap and lots of clients had overages under 95th. Even though 95th is an industry standard I feel it can be unfair at times. Which is why we adopted what we did. Best way to describe how its done is hybrid readings of your 95th and your actual to establish a fair average of those. We each eat something as a result, you never pay for full bw, and we never go looking for it all.

[wuurp]

Maybe it's counting the whole node as your bandwidth; perhaps it's counting backup as bandwidth?

I put this in Shared/SDX by mistake. This is a dedicated server.

Its not using an average, never has been. If your having those big bursts above 10mb quite often your going to drag that bw rate up. We go by the 95th standard , though to be honest I had devs drop it to 80th percent years ago since bw was so cheap and lots of clients had overages under 95th. Even though 95th is an industry standard I feel it can be unfair at times. Which is why we adopted what we did. Best way to describe how its done is hybrid readings of your 95th and your actual to establish a fair average of those. We each eat something as a result, you never pay for full bw, and we never go looking for it all.

"The Metered 95% MRTG stat is a moving average taken over the last 30 days, with the top 5% of the peak usage discarded." That sounds like an average to me.

I rarely have spikes to 10Mb/s. That one you see in the graph is unusual. If it weren't for my concern over this, I would be investigating to see who was doing that. I imagine someone sucked all the content from my server. And, I only see one perhaps 5 minute spike to maybe 5.5 for the latest day. So, that little spike raises my calculated BW to 49GB? Shouldn't that be in the "top 5% of the peak usage discarded"? That seems like an extremely unfair way of measuring bandwidth. Especially when according to the NIC the amount transferred was only about 20GB.

So, MRTG says I have used 1331 GB, and it will probably jump another 50 today. Even though actual bandwidth used will be about 18GB, and about 650GB for the month so far. You know, I'd be happy to pay for full bandwidth instead of this pretend bandwidth. I thought the point of metered bw is that while you do have a monthly limit, you aren't limited by spikes in traffic which you would be on a set speed. Well, seems that a few spikes is cutting my bandwidth in half. That certainly seems like it's being limiting to me.


Ok, I've been reading more about 95th percentile, and the way I understand it is the top 5% of the 5 minute measurements are discarded, then the very next highest rate is used to calculate your bw for the entire month. So, 2 days of high bandwidth, like a slashdot, and you're screwed. I had a couple busier days. So, it looks like I will be paying an overage charge even if I shut down the server right now, and even though I would only use about 750 GB of real bandwidth.

Yeah, I think this is one of those times it can be unfair.

[JPC-Greg]

No, your just not understanding how your bw is working and how 95th works. And yes that is how it works on being slashdotted for example. We have to make the bw there for you to use, when you use it you pay for it. As I said, we already took measures to bleed down the 95th so your not actually paying for 95th. I can tell you by looking at your chart and ym 10 yrs experience you would already be liable for 10mbps worth if we had true 95th in place.

[JPC-Greg]

Even at 1330 though and nearing the months end, your still below your allowance of 1500gb. You shouldnt worry about it too much. Even if you anticipate going over we can work with you to arrange some sweet bandwidth deals.

If your traffic fell to zero for the next few days it wouldnt go up. But you are adding to it daily, ever increasing the avg result for the months total. Hope that helps shed some light on why youll see it go up. Those spikes can be a costly burden if they go for too long. But like I said, if you get one or expect one, we can work with you on it. Dont panic

[JPC-Tracie]

Oh, drat. I meant for this to go in VPS, Dedicated.

*sneaks in*

I moved this thread to VPS & Dedicated for you

*sneaks out*

[wuurp]

I moved this thread to VPS & Dedicated for you

Thanks.

Even at 1330 though and nearing the months end, your still below your allowance of 1500gb. You shouldnt worry about it too much. Even if you anticipate going over we can work with you to arrange some sweet bandwidth deals.

If your traffic fell to zero for the next few days it wouldnt go up. But you are adding to it daily, ever increasing the avg result for the months total. Hope that helps shed some light on why youll see it go up. Those spikes can be a costly burden if they go for too long. But like I said, if you get one or expect one, we can work with you on it. Dont panic

It is slowing down, so maybe it will be ok.

You can't always tell when to expect a spike, so I thought I'd try capping the speed, at a level that will still allow good performance for visitors, but will limit the huge speed spikes.

I'm trying to get mod_cband to work in Apache, but no luck so far. No matter what limits I set as soon as it's enabled on a site, it says the site is over the limit. If I try to look at the status page it seg faults. This is on centos 5.x. I used an RPM from centos.karan.org. I would try a manual compile and install but can't find the source. The site seems to be gone.

Any suggestions, or suggestions for other methods to cap bandwidth?