1
I have to emulate a wide area network. My setup contains two servers with MT27500 Family [ConnectX-3] Infiniband cards. Default latency between both nodes is ~0.4ms and iperf test shows throughput of 4Gbps per iperf connection. I was able to get max of 33Gbps by using 15 parallel iperf connections. To simulate RRT of 50ms between both nodes I am using following command.
sudo tc qdisc add dev ib0 root netem delay 25ms
However iperf throughput perstream drops to 154Mbps. I have another identical setup which have Ethernet adapters instead of Infiniband. On that setup iperf shows around 480Mbps per iperf connection which is 3 times more than that I am getting from IPoIB. Is it expected behavioir that IPoIB peformes poor than ethernet over large latency? If so what is the reason behind this behavior?
Khayam Gondal
Posted 2018-07-27T20:26:55.577
Reputation: 111