0
I'm trying to simulate network latency, drops, etc between multiple datacenters (dc) using tc and iptables.
With iptables, I have configured each node to mark which dc a packet to traveling to: Here is an example of the mangle rules for 2 dcs (a1 and a2).
*mangle
-A POSTROUTING -j CLASSIFY --set-class 1:a1 -d 174.22.0.5
-A POSTROUTING -j CLASSIFY --set-class 1:a1 -d 174.22.0.6
-A POSTROUTING -j CLASSIFY --set-class 1:a2 -d 174.22.0.2
-A POSTROUTING -j CLASSIFY --set-class 1:a2 -d 174.22.0.4
COMMIT
So far so good (I think).
The problem comes when I enable tc rules (haven't even begun using netem yet):
tc qdisc del dev eth0 root
tc qdisc add dev eth0 parent root handle 1:0 htb default aff
tc class add dev eth0 parent 1:0 classid 1:a htb rate 1000kbit
tc class add dev eth0 parent 1:a classid 1:a1 htb rate 333kbit
tc class add dev eth0 parent 1:a classid 1:a2 htb rate 333kbit
tc class add dev eth0 parent 1:a classid 1:aff htb rate 333kbit
With this all the packets are being dropped:
tc -s qdisc show
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default aff direct_packets_stat 0 direct_qlen 0
Sent 0 bytes 0 pkt (dropped 15221, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
tc -s class show dev eth0
class htb 1:a2 parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b
Sent 0 bytes 0 pkt (dropped 7308, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 600593 ctokens: 600593
class htb 1:a root rate 1Mbit ceil 1Mbit burst 1600b cburst 1600b
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 200000 ctokens: 200000
class htb 1:aff parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b
Sent 0 bytes 0 pkt (dropped 1675, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 600593 ctokens: 600593
class htb 1:a1 parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b
Sent 0 bytes 0 pkt (dropped 7231, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 600593 ctokens: 600593
- Why is traffic being dropped?
- I don't necessarily need to do rate limiting, is there a better option?
Note: This example is using IPv4, but will need to also work for IPv6 which is why I've used iptables for classification.
1
try putting an sfq leaf qdisc.
– A.B – 2019-10-21T22:30:40.800As also described in https://lartc.org/howto - doesn't seem to work for me.
– kporter – 2019-10-21T23:08:06.863>tc class add dev eth0 parent 1:a1 handle a1: sfq perturb 10 Error: try "classid" instead of "handle"< – kporter – 2019-10-21T23:11:50.887
>tc class add dev eth0 parent 1:a1 classid a1: sfq perturb 10 Error: Qdisc "sfq" is classless.< – kporter – 2019-10-21T23:11:59.073
Woops, typed class instead of qdisc. Thanks!
I believe I had read that the defualt leaf is pfifo_fast - why didn't that work. – kporter – 2019-10-21T23:18:39.333
Seems I can substitute netem in place of sfq so I can simulate the network latency etc. – kporter – 2019-10-22T20:09:08.327