1
1
I want to fill a 10Gbps link with very small udp packets (64B).
I use the following command
for i in 1 2 3 4 5 6 7 8; do netperf -H 192.168.56.2 -p 12865 -t UDP_STREAM -l 60 -C -c -- -m 64 -s 16M -S 16M -R 1 & done
But I cannot get more than 250Mbps throughput. For small messages I saw that a thread called ksoftirqd is using 100% of a CPU core at the receiver. So my question is
- How to lower the cpu usage of ksoftirqd for small messages?
Current setting: I have two ubuntu machines (8 cores and 16 cores) with Intel X520-T2 NICs and kernel 3.11 connected with a Juniper 10G switch. These are the changes I have for sysctlcfg.conf
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 0
net.core.rmem_max = 268435456
net.core.wmem_max = 268435456
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
net.core.netdev_max_backlog = 250000
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_congestion_control = htcp
I also increased txqueue size of NIC
/sbin/ifconfig eth0 txqueuelen 10000
I looked for interrupt coalescing using rx-usecs parameter but increasing that worsened the throughput.