5

I have a IPSec tunnel between two Pfsense machines. Both machines are connected to a 100mbps symmetrical connection. The latency between the two routers is ~70ms. I'm using AES-GCM-128 and SHA1, both machines support hardware acceleration of AES and CPU usage remains < 5%. But I'm having the strangest problem..

Bandwidth peaks around 6MB/s, then gradually decreases to 2MB/s, then gradually increases again to 6MB/s. It's a predictable sine wave. How can I get my bandwidth more consistent?

I tried enabling/disabling compression (currently disabled), playing with the MSS-clamp and MTU settings (1500/1460 respectively) and there doesn't seem to be a difference.

When I download a file directly through the public internet, I get 11MB/s which is closer to my 100mbps max.

What are some things I can try?

ensnare
  • 2,132
  • 6
  • 23
  • 39
  • 1
    I'd recommend you to get a set of simultaneous packet captures. How does that bandwidth loss gets represented? TCP Window decreases in one side? Packet loss? Does the same behaviour applies when the bulk of the transfer goes the other way? What tools are you using to test? Did you try iPerf to avoid storage issues? – Pedro Perez Mar 02 '15 at 19:44
  • I wonder if the tunnel is rotating the key too often and exhausting the entropy pool or something? – Falcon Momot Sep 05 '15 at 23:05
  • A few questions: 1. Are you blocking ICMP? Some types/codes are required for path MTU discovery. If they are blocked, that can result in fragments and wasted bandwidth. (Problem is most likely to present itself in a tunneling protocol like IPSec or GIF) 2. Are you looking at CPU usage in the GUI, or are you logging in via SSH and running **top** from the command line? The GUI refreshes slowly, and may miss CPU spikes that occur. – Jason Stewart Nov 18 '15 at 13:18
  • 3. Is the IPSec 100 Mbps connection the same link as your public internet? Many ISPs rate limit in such a way to game the speedtest sites (giving you a speed-burst after a period of inactivity, then dialing it back down after a minute or so). – Jason Stewart Nov 18 '15 at 13:18
  • Do buffer or socket errors increment in `netstat -s` during the slowdown? Do locked socket counter increment on either side? [Here](https://tinyvpn.org/misc/watch_net.txt) is a function for watching the counters change. – Aaron Feb 05 '19 at 16:41

0 Answers0