Use tc and iptables to simulate latency between multiple datacenters

0

I'm trying to simulate network latency, drops, etc between multiple datacenters (dc) using tc and iptables.

With iptables, I have configured each node to mark which dc a packet to traveling to: Here is an example of the mangle rules for 2 dcs (a1 and a2).

*mangle
-A POSTROUTING -j CLASSIFY --set-class 1:a1 -d 174.22.0.5
-A POSTROUTING -j CLASSIFY --set-class 1:a1 -d 174.22.0.6
-A POSTROUTING -j CLASSIFY --set-class 1:a2 -d 174.22.0.2
-A POSTROUTING -j CLASSIFY --set-class 1:a2 -d 174.22.0.4
COMMIT

So far so good (I think).

The problem comes when I enable tc rules (haven't even begun using netem yet):

tc qdisc del dev eth0 root
tc qdisc add dev eth0 parent root handle 1:0   htb default aff
tc class add dev eth0 parent 1:0 classid 1:a   htb rate 1000kbit
tc class add dev eth0 parent 1:a classid 1:a1  htb rate 333kbit
tc class add dev eth0 parent 1:a classid 1:a2  htb rate 333kbit
tc class add dev eth0 parent 1:a classid 1:aff htb rate 333kbit

With this all the packets are being dropped:

tc -s qdisc show
qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default aff direct_packets_stat 0 direct_qlen 0
 Sent 0 bytes 0 pkt (dropped 15221, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
tc -s class show dev eth0
class htb 1:a2 parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b 
 Sent 0 bytes 0 pkt (dropped 7308, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 600593 ctokens: 600593

class htb 1:a root rate 1Mbit ceil 1Mbit burst 1600b cburst 1600b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 200000 ctokens: 200000

class htb 1:aff parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b 
 Sent 0 bytes 0 pkt (dropped 1675, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 600593 ctokens: 600593

class htb 1:a1 parent 1:a prio 0 rate 333Kbit ceil 333Kbit burst 1599b cburst 1599b 
 Sent 0 bytes 0 pkt (dropped 7231, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 600593 ctokens: 600593
  1. Why is traffic being dropped?
  2. I don't necessarily need to do rate limiting, is there a better option?

Note: This example is using IPv4, but will need to also work for IPv6 which is why I've used iptables for classification.

kporter

Posted 2019-10-21T22:07:24.610

Reputation: 101

1

try putting an sfq leaf qdisc.

– A.B – 2019-10-21T22:30:40.800

As also described in https://lartc.org/howto - doesn't seem to work for me.

– kporter – 2019-10-21T23:08:06.863

>tc class add dev eth0 parent 1:a1 handle a1: sfq perturb 10 Error: try "classid" instead of "handle"< – kporter – 2019-10-21T23:11:50.887

>tc class add dev eth0 parent 1:a1 classid a1: sfq perturb 10 Error: Qdisc "sfq" is classless.< – kporter – 2019-10-21T23:11:59.073

Woops, typed class instead of qdisc. Thanks!

I believe I had read that the defualt leaf is pfifo_fast - why didn't that work. – kporter – 2019-10-21T23:18:39.333

Seems I can substitute netem in place of sfq so I can simulate the network latency etc. – kporter – 2019-10-22T20:09:08.327

No answers