I got a strange GRE perfomance issue.
The setup is shown bellow.
[host1] --- GRE over Eth 1G --- [linux router] --- Eth 10G --- [host2]
Iperf transfer rate host1>host2 reaches 950Mbps. It encreases lineary with several hosts on the left running in parallel, up to several Gbps.
But for the direction host2>host1 iperf transer rate is limited to ~400Mbps both for single and multiple parallel tests.
It is notable that:
the observed tcp window size is under 64K with very low retransmition rate.
plenty of packets emmited by the router with ethertype=0x0011 on 10G link . I suppose it is some type of back pressure.
I suspect that GRE routing stack uses very small internal shared buffer (64K) but I could not find any usefull info on it.
I'll be grateful for any advice how to fix this problem.
I use Centos 7 (kernel 3.10.0-693) on the router.
==
[wk@host2 ~]$ iperf3 -c host1 -t 100
Connecting to host host1, port 5201
[ 4] local host1 port 33930 connected to host2 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 79.7 MBytes 668 Mbits/sec 10 62.8 KBytes
[ 4] 1.00-2.00 sec 68.9 MBytes 578 Mbits/sec 1 5.46 KBytes
[ 4] 2.00-3.00 sec 43.7 MBytes 366 Mbits/sec 5 21.8 KBytes
[ 4] 3.00-4.00 sec 49.4 MBytes 414 Mbits/sec 2 28.7 KBytes
[ 4] 4.00-5.00 sec 52.6 MBytes 442 Mbits/sec 1 43.7 KBytes
reading from file out.pcap, link-type LINUX_SLL (Linux cooked)
13:08:37.355114 IP host2.33197 > host1.5201: Flags [S], seq 3986495932, win 14600, options [mss 1460,sackOK,TS val 3358395421 ecr 0,nop,wscale 9], length 0
13:08:37.355120 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 80:
0x0000: 0000 0800 4500 003c 15a1 4000 4006 9d18 ....E..<..@.@...
13:08:37.355642 In 00:1b:21:bc:5d:96 ethertype Unknown (0x0011), length 80:
0x0000: 0000 0800 4500 003c 0000 4000 3f06 b3b9 ....E..<..@.?...
13:08:37.355642 IP host1.5201 > host2.33197: Flags [S.], seq 1075451774, ack 3986495933, win 13980, options [mss 1410,sackOK,TS val 3173856449 ecr 3358395421,nop,wscale 8], length 0
13:08:37.355675 IP host2.33197 > host1.5201: Flags [.], ack 1, win 29, options [nop,nop,TS val 3358395421 ecr 3173856449], length 0
13:08:37.355678 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 72:
0x0000: 0000 0800 4500 0034 15a2 4000 4006 9d1f ....E..4..@.@...
0x0010: c1e8 0218 c1e8 021a 81ad 1451 ed9d 19bd ...........Q....
13:08:37.355720 IP host2.33197 > host1.5201: Flags [P.], seq 1:38, ack 1, win 29, options [nop,nop,TS val 3358395421 ecr 3173856449], length 37
13:08:37.355725 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 109:
0x0000: 0000 0800 4500 0059 15a3 4000 4006 9cf9 ....E..Y..@.@...
0x0010: c1e8 0218 c1e8 021a 81ad 1451 ed9d 19bd ...........Q....