On VPN, ping works only for first packet and cannot ssh between machines

Question

I'm setting up some cameras for remote viewing. Each camera is connected to a Raspberry Pi (running the latest version of raspian). I have one camera at my house ("Site A") and one at my parents' house ("Site B") so far. I also have a third site ("Site C") that has a static IP and a VPN appliance that I use as a hub. The specific connections are:

• Site A <=> Site C via IPsec tunnel. Site A is running a StrongSwan VPN server on a lubuntu vitual machine that's hosted on a Windows 10 machine. Site B has a VPN router. There are two tunnels running here: One connects the Site A subnet to the Site C local subnet. The second connects the Site A subnet to the subnet that Site C defines for any call-in Road Warriors.
• Site B <=> Site C via IPsec/L2TP in a "Road Warrior" configuration. This means that only the single Raspberry Pi with the camera is on the VPN from Site B.
• There is no "direct" connection between Site A and Site B.

Each site has its own subnet: Site A is 192.168.1.0/24, Site B (a single machine, as seen on the VPN) is 192.168.2.1/32, and Site C is 192.168.3.0/24. Routing tables are set up as needed on machines involved to pass through the VPN.

Both Raspberry Pis are connected via wireless. The one at Site B shows high signal quality (68/70) and the one at Site A shows mid-to-poor quality (40/70).

From a Windows 10 machine on the Site A subnet (wired connection), I can ssh into both Pis using the setup. This works great to both machines - Really no noticeable lag there. From that same Windows 10 machine, using Internet Explorer I can access a webpage hosted on either machine. (On B, I have my own webpage set-up. On A so far I have the Apache test page.) That loads a little slowly where there are images, but ok given the wireless connection and the performance of the pi.

Now I'm trying to set-up a mirror of the website at Site B on the Pi at Site A. I started by trying to scp configuration files directly from B to A, and that was hanging. Then I tried several experiments to see what is wrong:

1. Ping from Pi on A to B. This typically works on the first packet to go out and fails on all subsequent packets. If I stop the ping and start again immediately, all of the packets on the second attempt fail. If I wait a bit between attempts, then it's back to the original behavior where the first packet makes it and the subsequent packets are lost.
2. Ping B to Pi on A. Same as #1.
3. Run traceroute from Pi on A to B. This passes through a sensible number of hops, and the hops that are identified (i.e. don't appear as *.*.*.*) look right.
4. Run traceroute from B to Pi on A. Same as #3.
5. Run wget on Pi on A to retrieve the webpage hosted on B. This hangs for a long time, sometimes fails, but if I let it run long enough (I went to bed and came back) it will apparently get it eventually.
6. Run wget on B to retrieve a webpage hosted on Pi on A. Same as #5. (At this point the Pi on A is just showing the Apache start page.)
7. I looked for duplicated IP addresses, and I don't see any on any of the subnets.
8. Run wget on each Pi to retrieve the home page from a major news website. Both Pis can complete this task almost instantly.
9. Ping from the Windows 10 machine on Site A to both Pis. This works to both Pis, i.e. no packets are lost.

When I saw the poor wireless connection to the Pi at Site A, I thought that might explain everything, but now I'm not at all convinced. The poor connection might explain 5-6 above and be consistent with 3-4, but I don't think that it explains 1-2 or 9. It also doesn't explain why it all works so well from the wired, Windows machines on subnet A, nor does it account for how #8 completes so quickly.

I did find several threads on SE and other sites that describe behavior like 1-2, but, as best I could tell, the causes identified there do not apply to my situation. In particular, I appear to definitely have a route between the machines and I don't appear to have any IP address conflicts.

I can think of several work arounds for this problem, but I'd like to understand the behavior here as it is. Does anyone have an idea of what might be wrong here? Is it just the poor wireless connection to the one machine? If so, how does that jive with the things that do work? If not the wireless connection strength, what else might cause this?

UPDATE

I moved the Pi at Site A to a wired connection. This should eliminate any issues from a poor WiFi connection. That does not seem to change the behavior.

I also left ping from A to B running for quite a while. I get the strange behavior that every 300th packet works and all others are dropped. For example, packets 1, 301, 601, etc. I let it go up to 1201. This behavior is apparently repeatable.

Answer 1

I finally solved this by changing the settings on the (virtual) Linux machine hosting the VPN server at Site A. I had to change one of the system parameters:

echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects

While this works and therefore solves the problem (so far!), I don't understand why everything works using Windows clients and fails using Linux clients. If anyone comes along with that broader answer, I would love to see it!

For completeness, you also need these, I believe:

echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/all/accept_redirects

In my case, those values were already set as such. You can also make these permanent by editing /etc/sysctl.conf to reflect the same values as above on net.ipv4.ip_forward, net.ipv4.conf.all.accept_redirects, and net.ipv4.conf.all.send_redirects.

Note: The changes above need to be made with root access.

I did the command line version with echo first and then reloaded everything, meaning I did sudo sysctl --system, followed by sudo IPsec restart, followed by the commands to initiate my specific connections. I'm not sure if all of these "restarts" were necessary though.

I found most of this information on this site, which also contains additional information that might help someone else on a different problem.