Linux box as network gateway changes source address

I have a Ubuntu Server box(A) with an IPSec tunnel to another datacenter(AWS, through a VPC VPN). The tunnel is fine and I can ping the other side of the tunnel.

The problem is when I try to communicate with any host on the other side of the tunnel.

Suppose I have the following config:

Box A: 172.31.0.5

Box B: 172.30.0.5

If I do the following:

A> ping 172.30.0.5

The ICMP packets arrive in the target server(B), but their source address is not the IP of the source server(A).

Instead, they arrive with 169.254.248.xxx as the source address which is the private address of the IPSec tunnel within AWS VPC.

As 169.254.248.xxx does not exist outside box A, box B can't send a reply and the packets are lost.

I certain that this is a problem with my routing setup on server A, how can I debug this to find a solution?

greenboxal

Posted 2015-07-20T21:16:20.447

Reputation: 111

What do you mean by "169.254.248.xxx does not exist outside box A"? You said that 169.254.248.xxx is the address range of the IPSec tunnel, so that address existing in the tunnel. Box A has multiple IPs, 172.31.0.5 and 169.254.248.xxx. The networking stack is picking the best address to get to 172.30.0.5, which is the VPN IP. – heavyd – 2015-07-20T21:20:52.213

The counterpart in AWS for 169.254.248.xxx is not on the box B, but in AWS infrastructure. The IPSec tunnel is not between A and B but between A and B's network router. – greenboxal – 2015-07-20T21:28:24.540

Tried this. No reply in the ping command. I got the following on tcpdump: https://gist.github.com/greenboxal/7e7dea68037c736e1907

– greenboxal – 2015-07-20T21:38:54.810

@greenboxal, and what's wrong? Isn't the 3rd line on "Box A" the ping reply coming back?? – heavyd – 2015-07-20T21:41:03.773

The ping command doesn't receives the reply. This is the log: https://gist.github.com/greenboxal/71fe53cf134ea52465e0

– greenboxal – 2015-07-20T21:42:17.207

@greenboxal, do you know what the IP 172.30.10.214 is? – heavyd – 2015-07-20T21:49:47.677

Sorry, this IP is 172.30.0.5, I've just changed on the question to a simple one. – greenboxal – 2015-07-20T21:55:15.257

Hmm, everything in the dumps looks right. – heavyd – 2015-07-20T21:56:39.303

Linux box as network gateway changes source address

Answers