Shaping to your internet bandwidth

[Dieselboy]

To put this simply, I have a 10M connection and so I was shaping to 10M (inbound and outbound on the lan facing connection). The idea for the shaping outbound is a simple one, so that we dont send traffic greater than our ISP bandwidth so that the ISP policer does not come into play and start dropping our traffic. Similarly, shaping to the same CIR towards the LAN for the same reason although achieved slightly differently.
We use VOIP and webex meetings all day long - my goal was to ensure that these services work well through the internet between offices.

What I have found is that shaping to 10M is too high and so the ISP policer was still coming into play and therefore dropping high priority traffic. I reduced the shaping bandwidth to 9.5M and the issue was better but not resolved (voice quality improved but you could hear there were still some dropped packets. I next reduced the shaper to 9MB and immediately, issue resolved. So now, as we're shaping to less than the value at which the ISP policer comes into force, I can queue / shape traffic but prioritise voice and this traffic will pass out to the internet without being dropped by the ISP to rate-limit us to our CIR which is 10M.

I was a bit shocked at the low value of 9MB. I have not tried 9,1 / 9.2 / 9.3M etc but it's possible that I could go up very slightly without invoking the ISP policer. I found out that the ISP uses CDMA for some of our link. I have not yet queried with them (because we're moving away from them) but I assume that our 10M connection has included some overhead which is outside of our data traffic. Eg - the ISP is policing our traffic to 10M but this traffic has additional overhead (may be going across the CDMA portion) and so our real data is somewhat lower than 10M.

So this got me thinking, how do you really know what your bandwidth is? You could run internet speed tests but these are usually pretty vague. If I'm paying for 10M I am happy with about 9M real throughput as slightly less than the CIR on a speedtest is expected.

thoughts?

[icecream-guy]

are you/they using round numbers?  or specific  number of bytes.

I wonder if the ISP id doing something like hard drive manufacturers do, whereas selling 10MB, capping at 10,000KB,  which is 9.765625MB
(Typically the lower rounded numbers are used in configs to pad in for some overhead, 0.234375MB in this case)

This kinda fits in with what you are describing.

[deanwebb]

I recall that shaping allows for some overage beyond the set limit. Shaping to 10M would allow for small bursts above 10, which would get whacked. 9M means that the bursts are within the range of the 10M hard limit on the policer.

[routerdork]

This is a good doc for comparison.
http://www.cisco.com/c/en/us/support/docs/quality-of-service-qos/qos-policing/19645-policevsshape.html

Something else to keep in mind are the byte sizes of the packets. They can affect how the policy works. On some platforms a smaller byte size packet can actually far exceed the policer. On most Cisco platforms I've found it to be fairly accurate though.

[Reggle]

Before blaming the ISP, keep in mind two side effects:
1) As stated by deanwebb, there is a burst value on Cisco routers and likely other platforms as well. You may very well go above the configred value. A raw packet capture can sort that out for you.
2) If you, like me, are using a switch or ASIC-based platform for shaping: everything that is put in the priority queue in a Cisco Catalyst is transmitted as soon as it is put n the queue, regardless of shaping. If you shape to 10 Mbps on a Catalyst, it will do that except fr priority traffic. That will go up to the actual bandwidth of 100 Mbp or 1 Gbps.

If you're sure the above two aren't in play, it's the ISP alright.

[Dieselboy]

Good points! I took it that the max shaped bandwidth on the Riverbed would actually be the max bandwidth. I've raised a case to clarify this for more info as the manual does not give any indication of burst traffic. It does say max bandwidth but I wonder if the Riverbed shapes non-priority traffic and just ignores the priority traffic or switches it first before any shaping. If this happens though, and I'm really sending 10M worth of non-voip-traffic I could have 1 or 2MB of VOIP traffic in addition, so this would exceed the bandwidth by quite a bit. So I hope this is not the case else I'd have to pre-empt any possible scenario :-s

I was setting the bandwidth in the Riverbed to 10,000kbps (not 10240kbps) and now 9mb = 9,000kbps.

[Dieselboy]

Riverbed came back to me and stated the max bandwidth will not be exceeded.

[deanwebb]

Riverbed came back to me and stated the max bandwidth will not be exceeded.

Just had to say that, in case they're found to be going, "Huh! Didn't expect that!" or "Hang on, I see what's going on here..." at some point in the future.