I have a Linux box handling routing for my network.
Here are some relevant configurations (public IP Addresses masked to 192.0.0.x):
IP configuration:
[root@gw ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0d:b9:32:fa:d0 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20d:b9ff:fe32:fad0/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc fq_codel master br0 state DOWN group default qlen 1000
link/ether 00:0d:b9:32:fa:d1 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 00:0d:b9:32:fa:d2 brd ff:ff:ff:ff:ff:ff
inet6 fe80::20d:b9ff:fe32:fad2/64 scope link
valid_lft forever preferred_lft forever
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 00:0d:b9:32:fa:d1 brd ff:ff:ff:ff:ff:ff
inet 192.0.0.2/30 brd 192.0.0.3 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::20d:b9ff:fe32:fad1/64 scope link
valid_lft forever preferred_lft forever
(br0 is a bridge between eth1 and eth2. Currently there's no connection into to eth1.)
Routing table:
[root@gw ~]# ip route
default via 192.0.0.1 dev br0
192.0.0.0/30 dev br0 proto kernel scope link src 192.0.0.2
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1
Test route:
[root@gw ~]# ip route get 192.168.1.2
192.168.1.2 dev eth0 src 192.168.1.1
cache
Packet capture of a PING:
(as seen from PING)
[root@gw ~]# ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.476 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.701 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=0.466 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=0.701 ms
^C
--- 192.168.1.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.466/0.586/0.701/0.115 ms
(as seen from tcpdump on same machine)
[root@gw ~]# tcpdump -i eth0 -n host 192.168.1.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:48:58.187862 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 1, length 64
12:48:58.188242 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 1, length 64
12:48:59.187086 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 2, length 64
12:48:59.187696 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 2, length 64
12:49:00.188837 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 3, length 64
12:49:00.189214 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 3, length 64
12:49:01.187834 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 4, length 64
12:49:01.188450 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 4, length 64
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel
PING capture on machine 192.168.1.2:
[admin@raspberry ~]$ sudo tcpdump -i eth0 -n icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:51:32.586016 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 1, length 64
12:51:32.586822 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 1, length 64
12:51:33.587715 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 2, length 64
12:51:33.588446 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 2, length 64
12:51:34.589160 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 3, length 64
12:51:34.589885 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 3, length 64
12:51:35.590523 IP 192.0.0.2 > 192.168.1.2: ICMP echo request, id 1451, seq 4, length 64
12:51:35.591254 IP 192.168.1.2 > 192.0.0.2: ICMP echo reply, id 1451, seq 4, length 64
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel
[admin@raspberry ~]$
Ok, so the PING does work, but my question remains: why is Linux assigning the source IP of the public facing interface to pings destined for the internal interface?
The problem became evident when I had a test machine I was setting up on the internal LAN that deliberately did not have a default gateway. Thus, PINGs were going to the machine sourced from 192.0.0.2 and thus the machine's replies never went through to the server (because it didn't have a default gateway configured).
PINGs going into the gateway box do come back properly, most likely because their source IPs are set correctly and so the PING reply coming from the gateway is using the correct source IP for the reply.
Given that the routing table shows the correct source, can anyone explain why the source IP is wrong, and how to fix it?
BTW:
[root@gw ~]# uname -a
Linux gw 4.0.4-2-ARCH #1 SMP PREEMPT Fri May 22 03:19:32 UTC 2015 i686 GNU/Linux
