3

It is all good on windows, but on linux when I try to retrieve a specific web page, I get a long wait and then a "connection reset by peer"

Pinging destination IP works fine.

I tried to reduce interface MTU to 1476(found using "ping -c1 -M do -s") and even lower values but it did not solve the problem.

On another linux PC near the destination host, there is no problem, so i suspect some router in the path.

These are wireshark and tshark output:

Linux with connection reset: http://pastebin.com/tpjS5qZc

Windows with no problem: http://pastebin.com/iyN1GDxT

It seems that third packet gets lost in the path to destination host and destination sends back several duplicate ack packets, but i can not see any relevant difference in windows and linux packets.

Mohammad Salehe
  • 133
  • 1
  • 1
  • 4

1 Answers1

9

In your capture both servers are setting "Do not fragment bit". This means that both ends are trying to do Path MTU Discovery.

It seems that there is a firewall that blocks ICMP Fragmentation Needed form your Linux server towards the remote server. As a workaround enable MSS clamping with: 

iptables -A OUTPUT -p tcp --tcp-flags SYN,RST SYN -j TCPMSS  --clamp-mss-to-pmtu

You can also try to disable P MTU Discovery in Linux:

echo  1  |sudo tee /proc/sys/net/ipv4/ip_no_pmtu_disc

From the iptables man page:

TCPMSS This target allows to alter the MSS value of TCP SYN packets, to control the maximum size for that connection (usually limiting it to your outgoing interface's MTU minus 40 for IPv4 or 60 for IPv6, respectively). Of course, it can only be used in conjunction with -p tcp.

   This  target is used to overcome criminally braindead ISPs or servers which block "ICMP Fragmentation Needed" or "ICMPv6 Packet Too
   Big" packets.  The symptoms of this problem are that everything works fine from your Linux firewall/router, but machines behind  it
   can never exchange large packets:
    1) Web browsers connect, then hang with no data received.
    2) Small mail works fine, but large emails hang.
    3) ssh works fine, but scp hangs after initial handshaking.
   Workaround: activate this option and add a rule to your firewall configuration like:

           iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN
                       -j TCPMSS --clamp-mss-to-pmtu

   --set-mss value
          Explicitly  sets  MSS  option  to  specified  value.  If  the  MSS of the packet is already lower than value, it will not be
          increased (from Linux 2.6.25 onwards) to avoid more problems with hosts relying on a proper MSS.

   --clamp-mss-to-pmtu
          Automatically clamp MSS value to (path_MTU - 40 for IPv4; -60 for IPv6).  This may not function as desired where  asymmetric
          routes  with  differing  path MTU exist — the kernel uses the path MTU which it would use to send packets from itself to the
          source and destination IP addresses. Prior to Linux 2.6.25, only the path MTU to the destination IP address  was  considered
          by this option; subsequent kernels also consider the path MTU to the source IP address.

   These options are mutually exclusive.

See: http://lartc.org/howto/lartc.cookbook.mtu-mss.html

Edit: After I've take a closer look on the captures, I've discovered that there is a broken firewall along the path that is filtering all IP packets that use TCP Timestamp option. Just run on the Linux box: echo 0 | sudo tee /proc/sys/net/ipv4/tcp_timestamps

Mircea Vutcovici
  • 16,706
  • 4
  • 52
  • 80
  • I did both setting no_mtu_disc and MSS clamping. The only change that I see is DF flag not being set on outgoing packets: http://pastebin.com/gfKaznT5 – Mohammad Salehe Jul 27 '12 at 07:58
  • This was the same when I tested from different linuxes across the world, but there was no problem from the destination country(Iran). Could be this some kind of intentional filtering?! I have no problem with other web sites in Iran it seems. – Mohammad Salehe Jul 27 '12 at 08:02
  • Try also with `--set-mss`, because you have great chance of having an asymmetric route. This means that the forward path goes trough some routers and part of the return path goes trough other routers. – Mircea Vutcovici Jul 27 '12 at 16:26
  • I think it is network misconfiguration somewhere on the path, not intentional filtering. – Mircea Vutcovici Jul 27 '12 at 16:28
  • I had tried that, did not help. Is there some way for further inspection? – Mohammad Salehe Jul 27 '12 at 21:20
  • Can you also try with a HTTP request that will return a very small page. E.g. start with a page that do not exists and should return a 404. Or a page that returns a redirection. Find a page like this with the Windows box. – Mircea Vutcovici Jul 27 '12 at 21:54
  • Actually the page I am trying is a small redirect. ok, I'm trying to get this page: http://ime.co.ir/ – Mohammad Salehe Jul 28 '12 at 06:57
  • 1
    It is a broken firewall along the path. Just run on the Linux box: `echo 0 | sudo tee /proc/sys/net/ipv4/tcp_timestamps` – Mircea Vutcovici Jul 29 '12 at 03:45
  • See also: http://prowiki.isc.upenn.edu/wiki/TCP_tuning_for_broken_firewalls – Mircea Vutcovici Jul 29 '12 at 03:47
  • Welcome ;) I will try using http://www.dirk-loss.de/scapy-doc/usage.html to figure out which router is doing this. I will try to send the GET request packet with a lower TTL. – Mircea Vutcovici Aug 01 '12 at 14:39
  • 2
    Thanks! You've saved my day with `echo 1 | sudo tee /proc/sys/net/ipv4/ip_no_pmtu_disc`. I had a problem with DB2 communication freezing for long SQL statements (> 3000 chars). Symptoms at http://www-01.ibm.com/support/docview.wss?uid=swg21590014. – Martin Ždila Oct 02 '15 at 12:32