5

I have been trying to perform VLAN based rate limiting on an academic project I am working on. The project uses Openflow (with Beacon controller) and the network is emulated using Mininet. The network is made up of OpenVSwitches.

For my project I need to rate-limit the traffic on each VLAN in the network. I have chosen tc (with htb) for this and have been trying from many days to get this working. These are the goals I want to achieve using tc: 1) Rate limit all VLANs with a certain data rate. 2) Make sure that once the rate limits are set, all VLANs using a physical link between switches get the same (or nearly same) maximum data rate.

So if I set the data rate limit on each of the VLANs as 100 Mbps and there are 5 VLANs using a physical link, I want all these VLANs NOT to exceed 100Mbps and at the same time get maximum data rates of around 100 Mbps each.

I have written script for this topology:

s2---s4
|
s6---s8
|
s10---s12

This is that script:

#!/bin/bash

# delete prev: 
tc qdisc del dev s10-eth2 root
tc qdisc del dev s2-eth3 root
tc qdisc del dev s6-eth2 root
tc qdisc del dev s6-eth3 root
tc qdisc del dev s6-eth4 root
tc qdisc del dev s8-eth2 root

# create "roots":
tc qdisc add dev s10-eth2 root handle 1: htb default 1000
tc class add dev s10-eth2 parent 1: classid 1:1 htb rate 1000Mbit ceil 1000Mbit
tc qdisc add dev s2-eth3 root handle 2: htb default 1001
tc class add dev s2-eth3 parent 2: classid 2:1 htb rate 1000Mbit ceil 1000Mbit
tc qdisc add dev s6-eth2 root handle 3: htb default 1002
tc class add dev s6-eth2 parent 3: classid 3:1 htb rate 1000Mbit ceil 1000Mbit
tc qdisc add dev s6-eth3 root handle 4: htb default 1003
tc class add dev s6-eth3 parent 4: classid 4:1 htb rate 1000Mbit ceil 1000Mbit
tc qdisc add dev s6-eth4 root handle 5: htb default 1004
tc class add dev s6-eth4 parent 5: classid 5:1 htb rate 1000Mbit ceil 1000Mbit
tc qdisc add dev s8-eth2 root handle 6: htb default 1005
tc class add dev s8-eth2 parent 6: classid 6:1 htb rate 1000Mbit ceil 1000Mbit

# classes for all vlans (ie: VLAN ids 541, 542 and 543) on each interface:
tc class add dev s10-eth2 parent 1:1 classid 1:541 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s10-eth2 parent 1:1 classid 1:542 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s10-eth2 parent 1:1 classid 1:543 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s2-eth3 parent 2:1 classid 2:542 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s6-eth2 parent 3:1 classid 3:542 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s6-eth3 parent 4:1 classid 4:543 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s6-eth4 parent 5:1 classid 5:541 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s6-eth4 parent 5:1 classid 5:542 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s6-eth4 parent 5:1 classid 5:543 htb rate 100Mbit ceil 100Mbit prio 1
tc class add dev s8-eth2 parent 6:1 classid 6:543 htb rate 100Mbit ceil 100Mbit prio 1

# parent filter:
tc filter add dev s10-eth2 parent 1: prio 1 protocol 802.1q u32
tc filter add dev s2-eth3 parent 2: prio 1 protocol 802.1q u32
tc filter add dev s6-eth2 parent 3: prio 1 protocol 802.1q u32
tc filter add dev s6-eth3 parent 4: prio 1 protocol 802.1q u32
tc filter add dev s6-eth4 parent 5: prio 1 protocol 802.1q u32
tc filter add dev s8-eth2 parent 6: prio 1 protocol 802.1q u32

# filter for each VLAN (with VLAN ids 541, 542 and 543):
tc filter add dev s10-eth2 parent 1: prio 1 protocol 802.1q u32 match u16 0x021D 0x0fff at -4 flowid 1:541
tc filter add dev s10-eth2 parent 1: prio 1 protocol 802.1q u32 match u16 0x021E 0x0fff at -4 flowid 1:542
tc filter add dev s10-eth2 parent 1: prio 1 protocol 802.1q u32 match u16 0x021F 0x0fff at -4 flowid 1:543
tc filter add dev s2-eth3 parent 2: prio 1 protocol 802.1q u32 match u16 0x021E 0x0fff at -4 flowid 2:542
tc filter add dev s6-eth2 parent 3: prio 1 protocol 802.1q u32 match u16 0x021E 0x0fff at -4 flowid 3:542
tc filter add dev s6-eth3 parent 4: prio 1 protocol 802.1q u32 match u16 0x021F 0x0fff at -4 flowid 4:543
tc filter add dev s6-eth4 parent 5: prio 1 protocol 802.1q u32 match u16 0x021D 0x0fff at -4 flowid 5:541
tc filter add dev s6-eth4 parent 5: prio 1 protocol 802.1q u32 match u16 0x021E 0x0fff at -4 flowid 5:542
tc filter add dev s6-eth4 parent 5: prio 1 protocol 802.1q u32 match u16 0x021F 0x0fff at -4 flowid 5:543
tc filter add dev s8-eth2 parent 6: prio 1 protocol 802.1q u32 match u16 0x021F 0x0fff at -4 flowid 6:543

And these are the statistics, after I execute this script and perform transmission:

qdisc htb 1: root refcnt 2 r2q 10 default 1000 direct_packets_stat 82134
 Sent 5425380 bytes 82134 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 2: root refcnt 2 r2q 10 default 1001 direct_packets_stat 306284
 Sent 462713848 bytes 306284 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 3: root refcnt 2 r2q 10 default 1002 direct_packets_stat 62794
 Sent 4144482 bytes 62795 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 4: root refcnt 2 r2q 10 default 1003 direct_packets_stat 0
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 5: root refcnt 2 r2q 10 default 1004 direct_packets_stat 831269
 Sent 1257477000 bytes 831270 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 6: root refcnt 2 r2q 10 default 1005 direct_packets_stat 0
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0

I am trying to set a data rate limit of 100 Mbps on all VLANs using this script. The rate limiting does not work at all, in fact they go haywire. The data rates which each of my hosts achieve is more than 300+ Mbps which way higher than what I have set, plus none of the data flows using the VLANs get uniform data rates.

Would be great if someone could please help me in getting this script rectified and working. Thanks.

user1449265
  • 151
  • 1
  • 2
  • perhaps the problem is where you're doing the rate limiting... the physical interface is fixed at 1Gbps or faster, but the L3 routing point is the routing engine upstream. More than likely, the host is not throttling at all and they are all overwhelming the routing engine. You may just have to go about this a different way. – SpacemanSpiff Dec 27 '12 at 14:14
  • @SpacemanSpiff: Thank you. Would be great if you could please suggest where else it is possible to do the rate limiting? – user1449265 Dec 27 '12 at 14:53
  • I don't know these technologies, but a diagram would be useful, for instance downstream on a switch do you want to restrict to 100Mb if it's going port to port, or only when headed out of the switch? What about from the same VLAN to the same VLAN between racks? – SpacemanSpiff Dec 27 '12 at 15:23
  • possible duplicate of [Limiting bandwidth on internal interface on Linux gateway](http://serverfault.com/questions/455561/limiting-bandwidth-on-internal-interface-on-linux-gateway) – Michael Hampton Dec 27 '12 at 16:26
  • @SpacemanSpiff: I am very inexperienced with tc and traffic shaping. But yeah, I want to rate limit only outgoing traffic. – user1449265 Dec 27 '12 at 18:29
  • @MichaelHampton: I do not use IP, I use 802.1q for matching. So I think it is quite different. – user1449265 Dec 27 '12 at 18:30
  • I found that different versions of the linux kernels treat 802.1q based matching differently. Apparently, kernel 3.0.0-12 and upwards do not support matching on 802.1q at all:http://serverfault.com/questions/369603/tc-u32-match-protocol-any-in-recent-kernels-where-to-discuss . Is this true? And is there a solution or a work around so far? – user1449265 Dec 27 '12 at 18:34
  • I found a fix. The problem was with the kernel. I now use Debian 5 (lenny) with kernel version 2.6.33. Everything works well. – user1449265 Dec 31 '12 at 10:20

1 Answers1

2

I made a script that shapes traffic on separate VLANs passing throw a bridge. I hope this can be useful for somebody searching for a solution.

#!/bin/bash
## in this example I use vlan 2001-2004, which are hex equivalents of 0x07D1 -0x07D4

# internet facing interface
INET=eth2
# downlink facing interface
DOWN=eth3

### configuring download rules:
tc qdisc del dev $DOWN root
tc qdisc replace dev $DOWN root handle 1: htb
tc class replace dev $DOWN parent 1: classid 1:2001 htb rate 11mbit
tc class replace dev $DOWN parent 1: classid 1:2002 htb rate 21mbit
tc class replace dev $DOWN parent 1: classid 1:2003 htb rate 31mbit
tc class replace dev $DOWN parent 1: classid 1:2004 htb rate 41mbit
tc filter replace dev $DOWN protocol all parent 1: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D1)" flowid 1:2001
tc filter replace dev $DOWN protocol all parent 1: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D2)" flowid 1:2002
tc filter replace dev $DOWN protocol all parent 1: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D3)" flowid 1:2003
tc filter replace dev $DOWN protocol all parent 1: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D4)" flowid 1:2004

printf "########## Printout for DOWNLOAD: ###################\n"
printf "\n"
printf "$(tc filter show dev $DOWN)\n"
printf "$(tc class show dev $DOWN)\n"
printf "$(tc qdisc show dev $DOWN)\n"
printf "\n"

### configuring upload rules:
tc qdisc del dev $INET root
tc qdisc replace dev $INET root handle 2: htb
tc class replace dev $INET parent 2: classid 2:2001 htb rate 11mbit
tc class replace dev $INET parent 2: classid 2:2002 htb rate 21mbit
tc class replace dev $INET parent 2: classid 2:2003 htb rate 31mbit
tc class replace dev $INET parent 2: classid 2:2004 htb rate 41mbit
tc filter replace dev $INET protocol all parent 2: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D1)" flowid 2:2001
tc filter replace dev $INET protocol all parent 2: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D2)" flowid 2:2002
tc filter replace dev $INET protocol all parent 2: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D3)" flowid 2:2003
tc filter replace dev $INET protocol all parent 2: prio 100 basic match "meta(vlan mask 0xfff eq 0x07D4)" flowid 2:2004

printf "########## Printout for UPLOAD: #####################\n"
printf "\n"
printf "$(tc filter show dev $INET)\n"
printf "$(tc class show dev $INET)\n"
printf "$(tc qdisc show dev $INET)\n"
printf "\n"

exit 0
Sergiu Rudenco
  • 21
  • 2