1

I asked a similar question before, but I believe I was not clear with the details.

I'm basically having some problems with my server related to high latency between the server and the clients due to the geographical distance. And making a research, it turned out that I can solve this issue by increasing the TCP Window Size.

The problem is I don't really know how to do it, and following the instructions I found on the web didn't help at all.

So I'm hoping that someone can share some documentation / commands / instructions or anything that can further improve this question.

Here is some info:

  • OS: Ubuntu 20.04 (LTS) x64
  • Server: Apache/2.4.41
  • Application Type: Python - Flask
  • Hosting Company: Digital Ocean

Edit: After several replies telling me that adjusting the Window Size won't solve the problem, I think I was not clear enough.

Please refer to this answer before telling me that changing the Window Size won't work. https://networkengineering.stackexchange.com/a/2297/71565

If you believe that I misunderstood the answer in the link above, please tell me what in particular I'm getting wrong and don't just share a cryptic reply of "it won't work" etc.

Efe Zaladin
  • 113
  • 5
  • What is the actual problem? – Michael Hampton Oct 18 '20 at 22:55
  • @MichaelHampton The problem is, when transmitting files with high volume, the files are broken into many parts and transferred with TCP. As you may know TCP protocol needs to do a handshake. So each TCP packet adds a delay equal to the RTT. For instance the equation that gives the approximate download time is, RTT*Size/64KB. The default window size is 64KB. If I can increase it to something like 1MB, this extra delay will vanish. – Efe Zaladin Oct 19 '20 at 06:19
  • TCP Window size has likely nothing to do with your issue. Please verify that by using `tcpdump` on a connection that is slow. When there is long geographical distance, there are likely slow / congested links between source and destination, which makes things slow. You should move your services clsoer to the end-user. – Tero Kilkanen Oct 19 '20 at 06:40
  • 1
    @TeroKilkanen The problem is caused by high latency (avg 70ms) and can be solved by increasing the window size. I've validated this conjecture experimentally by doing iperf tests with different window sizes. The results are; 64KB -- 150-300 Kb/s and 4MB -- 10-50 Mb/s. I think I'm clear on the reasoning behind this, which I shared in my first comment. If you know how to increase the TCP Window Size, I would be much appreciated if you share it. Then I can share the results with you. – Efe Zaladin Oct 19 '20 at 07:01
  • You do not fix a latency problem by changing the window size. The latency dues to distance is something you cannot fix, except by getting closer. – Ron Maupin Oct 19 '20 at 13:34
  • @RonMaupin What makes you think that increasing the windows size won't solve the problem despite the fact I have experimental results? – Efe Zaladin Oct 19 '20 at 15:04
  • 70ms is not high latency at all. What does the TCP Window size look like when you capture packets of a slow connection? – Tero Kilkanen Oct 19 '20 at 16:27

1 Answers1

1

I'm not 100% convinced your issue is really window-size related. Still, below you can find the relevant info.

The base window size can not exceed 65535 bytes due to limitations of the TCP header. From RFC 1323:

The TCP header uses a 16 bit field to report the receive window size to the sender. Therefore, the largest window that can be used is 2**16 = 65K bytes.

This does not means the TCP window can't be bigger, as modern OSes support (and advertise by default) TCP Window Scaling, where the window size is dynamically increased via scaling by up to 14X factor. However, an upper limit can be configured before hitting maximum scaling.

For reference, these are the relevant parameters (and their defaults) on Ubuntu 20.04:

net.ipv4.tcp_window_scaling = 1         ;scaling enabled
net.ipv4.tcp_rmem = 4096 131072 6291456 ;min, default and max receive window
net.ipv4.tcp_wmem = 4096  16384 4194304 ;min, default and max send window
net.core.rmem_max = 212992              ;max absolute limit for receive buffer
net.core.wmem_max = 212992              ;max absolute limit for send buffer

The actual maximum window size is the smaller between [r|w]mem_max and the third value of tcp_[r|w]mem. So on Ubuntu 20.04 you have an actual max receive and send window of 212992 bytes by default. To increase that limit to 4 MB you can do the following:

sysctl -w net.core.rmem_max=4194304
sysctl -w net.core.wmem_max=4194304

If it works, you can persist the settings by editing /etc/sysctl.conf

shodanshok
  • 44,038
  • 6
  • 98
  • 162