7

I have two machines (with Centos 7.3 on both) connected via 1G switch. When I measure UDP thoughput between macines with iperf it gives about 850 Mbit/s. However when I measure multicast UDP thoughput it results in something like 95 Mbit/s (almost 10x slowdown).

What are possible causes for that?

Sample output for unicast:

$ iperf -c 192.168.1.11 -u -b 9900m -f m -i 5 -t 30 -w 1m
------------------------------------------------------------
Client connecting to 192.168.1.11, UDP port 5001
Sending 1470 byte datagrams, IPG target: 1.19 us (kalman adjust)
UDP buffer size: 2.00 MByte (WARNING: requested 1.00 MByte)
------------------------------------------------------------
[  3] local 192.168.1.208 port 52738 connected with 192.168.1.11 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec   503 MBytes   845 Mbits/sec
[  3]  5.0-10.0 sec   507 MBytes   850 Mbits/sec
[  3] 10.0-15.0 sec   500 MBytes   839 Mbits/sec
[  3] 15.0-20.0 sec   499 MBytes   837 Mbits/sec
[  3] 20.0-25.0 sec   497 MBytes   834 Mbits/sec
[  3] 25.0-30.0 sec   501 MBytes   841 Mbits/sec
[  3]  0.0-30.0 sec  3008 MBytes   841 Mbits/sec
[  3] Sent 2145384 datagrams
[  3] Server Report:
[  3]  0.0-30.0 sec  3008 MBytes   841 Mbits/sec   0.406 ms    0/2145384 (0%)

Sample output for multicast:

$ iperf -c 239.255.1.3 -u -b 990m -f m -i 5 -t 30 -w 1m
------------------------------------------------------------
Client connecting to 239.255.1.3, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11.88 us (kalman adjust)
Setting multicast TTL to 1
UDP buffer size: 2.00 MByte (WARNING: requested 1.00 MByte)
------------------------------------------------------------
[  3] local 192.168.1.208 port 53248 connected with 239.255.1.3 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec  57.2 MBytes  95.9 Mbits/sec
[  3]  5.0-10.0 sec  57.0 MBytes  95.6 Mbits/sec
[  3] 10.0-15.0 sec  56.4 MBytes  94.6 Mbits/sec
[  3] 15.0-20.0 sec  56.4 MBytes  94.6 Mbits/sec
[  3] 20.0-25.0 sec  56.4 MBytes  94.6 Mbits/sec
[  3] 25.0-30.0 sec  54.5 MBytes  91.4 Mbits/sec
[  3]  0.0-30.0 sec   338 MBytes  94.3 Mbits/sec
[  3] Sent 240946 datagrams

Some observations:

  • The problem is symmetrical (switching positions of iperf client and server does not help)
  • According to iptraf-ng traffic on the sender's side is already low. However I don't know if I can trust it.

Upd: Switch: Netgear GS108 (unmanaged)

Alexei Osipov
  • 173
  • 6
  • 1
    What is the switch? Perhaps its backplane is a bottleneck. – EEAA May 04 '17 at 12:35
  • >What is the switch? Perhaps its backplane is a bottleneck I agree with this, the backplane is a regular bottleneck. – Net Runner May 04 '17 at 14:02
  • Bandwidth for this switch is 16 Gbps (according to the [spec](http://www.downloads.netgear.com/files/GDC/datasheet/en/GS105v5-GS108v4.pdf)). Seems like should be OK for 8 ports. Also I'm not sure if issue with switch can explain low traffic (95Mbit/s) on the sender's side. – Alexei Osipov May 04 '17 at 14:29
  • 6
    Do you have something else connected to the switch that could be slowing down the traffic. The bandwidth figures of 91-96 Mbit/s are suspiciously near 100 Mbit/s, maybe there is a port that is operating at 100 Mbit/s. Or this could be an inherent limitation of the switch, as the comments above suggest. Anyway, you could analyze the traffic (with Wireshark, e.g.) to see if the switch is sending pause frames to perform flow control. See [this article](https://www.ibm.com/support/knowledgecenter/en/SSQPD3_2.6.0/com.ibm.wllm.doc/ethernetflowcont.html). – Johan Myréen May 05 '17 at 12:40
  • Johan Myréen, thanks for the article! It was very helpful at explaining our observations. – Alexei Osipov May 05 '17 at 16:49
  • Do you have a slow port as suggested by Johan Myreen? – mm759 May 20 '17 at 12:35
  • 1
    Yes, Johan Myréen was right. Disconnecting uplink from the switch fixed the multicast speed issue. So when there are only two machines connected to the switch the multicast works at full speed. Btw, the uplink was 1Gbit too. Seems like flow control from uplink side was limiting speed to 100 Mbit. – Alexei Osipov May 23 '17 at 16:05

2 Answers2

1

Check whether the switch is using flow control. It'll issue pause frames from the source port when any linked port isn't able to keep up with the multicast rate, ie. is linked at 100 or 10 Mbit/s.

You might want to disable flow control for the test. I generally disable it since it's rarely useful and may cause head-of-line blocking, as in your case. TCP does a much better job and there are few cases where Ethernet flow control makes sense (e.g. in a properly designed iSCSI SAN).

Zac67
  • 8,639
  • 2
  • 10
  • 28
  • Yes, flow control was the cause. See also https://serverfault.com/questions/848226/why-multicast-udp-can-be-slower-than-unicast-udp/1093362#comment1088691_848226 – Alexei Osipov Apr 14 '22 at 20:29
0

You have a difference on the "-b" (bandwidth) parameter for iPerf. I am guessing that this is leading to the difference reported in IPG (inter-packet gap), which is a factor of 10 between your multicast and unicast cases.

Suspiciously close to your difference in reported bandwidth. So that might all there is to it.

Raul Benet
  • 98
  • 5