Using tc in Linux and dummynet in FreeBSD to limit bandwidth

We have a network product that runs on Linux and needs to be tested in reduced bandwidth networks.

We have been able to use 'tc' (traffic control) in Linux and dummynet in FreeBSD to inject delay (latency) with predictable behavior.

But when we try to throttle the bandwidth with either of these, it does not seem to have much effect.

Here is an example of trying to use 'tc' to set the bandwidth to 512Kbps:

tc qdisc change dev eth0 root tbf rate 512kbit burst 1540

But when we run 'scp' over that connection we see speeds that are way higher than we expect.

Here are results of setting various bandwidths in both dummynet and tc. You can see that it has SOME effect but not very much.

BW             _______DUMMYNET__________        __________ TC __________
10Mbps         1m,3.489s        4.0MB/s         1m,2.732s       4.1MB/s
8Mbps          1m,3.461s        4.0MB/          1m,2.728s       4.1MB/s
6Mbps          1m,3.905s        4.0MB/s         1m,2.710s       4.0MB/s
1Mbps          1m, 3.261s       4.0MB/s         1m,2.680s       4.0MB/s
512Kbps        1m,26.175s       3.0MB/s         1m 25.409s      3.0MB/s
256Kbps        2m,39.079s       1.6MB/s         2m 53.004s      1.5MB/s
128Kbps        5m, 3.501s       852.7KB/s       5m, 50.491s     739.7KB/s
64Kbps         9m,48.907s       463.3KB/s       9m,53.105s      436.5KB/s

I don't have the syntax we've used for DUMMYNET handy but I'll post it later.

lcbrevard

Posted 2010-12-03T16:57:07.853

Reputation: 568

This may solve our problem for 'tc' but if anyone knows how to setup dummynet that would be good to know. A potential customer needs to see this run in a dummynet based network emulation. Here is the post with the 'tc' info: http://superuser.com/questions/147156/simulating-a-low-bandwidth-high-latency-network-connection-on-linux

– lcbrevard – 2010-12-03T17:24:57.543

working on getting it working in vm (dummynet bridge) with a proof video...may take a bit to get it working record and post video, but it's coming – RobotHumans – 2010-12-03T17:50:38.633

note in dummynet you don't just ratelimit you also have a delay parameter – RobotHumans – 2010-12-07T13:28:10.483

Oops... I commented below before I saw this. The customer environment has reduced bandwidth AND very long delay (>500ms). We showed that the bandwidth reduction was simply not working with the above tests. If we only put in delay we get the expected results. We concluded that our setup for bandwidth reduction was wrong on both tc and dummynet! – lcbrevard – 2010-12-07T17:10:46.057

Answers

There is a question on serverfault that I answered with a more extensive example of using tc and iptables to do traffic limiting. This example comes from a setup we have on our mirror server and it works very well.

Sean Reifschneider

Posted 2010-12-03T16:57:07.853

Reputation: 1 387

I wouldn't run the filtering on the box in question, i would run it in a dedicated box/vm between the two pieces of tech in question. That way you can run a clean baseline and more adequately evaluate results(run ethereal/wireshark on both sides and see the size difference to make sure you're working properly)

here's something you might find useful: http://www.dummycloud.com/screenshots

although tc should do it for you

RobotHumans

Posted 2010-12-03T16:57:07.853

Reputation: 5 758

We've been using a dedicated system to run dummynet and a dedicated VM to run 'tc' but I believe we've had the syntax / order of commands wrong. – lcbrevard – 2010-12-05T17:19:16.720

is there a preference? with the work i am putting in to the VM with dummynet and video, i might be willing to turn it into live media for you – RobotHumans – 2010-12-05T20:14:23.157

I need to understand both scenarios. One of our potential customers prefers to use a dedicated system running FreeBSD 8.1-RELEASE with some kernel modifications and dummynet. My testing environment works easier using VMs for everything. I am getting much more familiar with tc but need to learn dummynet as well. – lcbrevard – 2010-12-07T17:07:45.110

okay no problem...since freebsd is REQUIRED i may need to start my vm over(dummynet is now available on debianized and I have little experience with BSD so it may take a bit longer....either way the livecd should prove useful...pop it in hardware and make sure you have goot network drivers.....done). – RobotHumans – 2010-12-07T19:35:21.700