2
1
I am using the below commands to setup test environment for traffic flow. I am using a bash shell for traffic source but would eventually be VM or container.
I don't understand where the traffic is being dropped and was hoping someone could help.
echo 1 > /proc/sys/net/ipv4/conf/all/log_martians
brctl addbr br0
brctl addbr br1
ip netns add nstest
ip link add veth-a type veth peer name veth-b
ip link add veth-c type veth peer name veth-d
ip link set veth-b netns nstest
ip netns exec nstest ip addr add 172.20.0.2/24 dev veth-b
ip netns exec nstest ip route add default via 172.20.0.1
ip netns exec nstest ip link set dev veth-b up
brctl addif br0 veth-a
brctl addif br0 veth-d
brctl addif br1 veth-c
ip addr add 172.20.0.1/24 dev br1
ip link set dev br0 up
ip link set dev br1 up
ip link set dev veth-a up
ip link set dev veth-c up
ip link set dev veth-d up
ip route flush cache
find /proc/sys -name rp_filter -exec sh -c "echo 0 > {}" \;
find /proc/sys -name rp_filter -print -exec sh -c "cat {}" \;
iptables -t nat -I POSTROUTING -s 172.20.0.0/16 ! -d 172.20.0.0/16 -j MASQUERADE
ip netns exec nstest bash
$ ping -c 1 172.20.0.1/24
$ ping -c 1 8.8.8.8
Two bridges are connected with veth interfaces and the GW is placed on BR1 to make traffic traverse the two bridges.
172.20.0.2[veth-b]----[veth-a][br0][veth-d]-----[veth-c][br1][172.20.0.1]
From bash shell i can ping GW 172.20.0.1
OK but if i try to ping public address e.g. 8.8.8.8
i get no reply.
conntrack shows the traffic
icmp 1 28 src=172.20.0.2 dst=8.8.8.8 type=8 code=0 id=6085 [UNREPLIED] src=8.8.8.8 dst=10.0.2.15 type=0 code=0 id=6085 mark=0 use=1
The tcpdump is bit weird. The MAC e2:1c:84:b1:a3:5f
is assigned to veth-d interface in the nstest network namespace.
12:37:53.547319 P e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 172.20.0.2 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547435 Out e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547437 In e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547437 In e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 0, seq 1, length 64
And this is a iptables TRACE
TRACE: raw:PREROUTING:policy:2 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:PREROUTING:policy:1 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:PREROUTING:policy:2 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:FORWARD:policy:1 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:FORWARD:rule:1 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:DOCKER-ISOLATION:return:7 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:FORWARD:policy:16 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:POSTROUTING:policy:2 IN= OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:POSTROUTING:rule:1 IN= OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: raw:PREROUTING:policy:2 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:PREROUTING:policy:1 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:PREROUTING:policy:2 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
IPv4: martian source 8.8.8.8 from 10.0.2.15, on dev br1
ll header: 00000000: 02 e4 cc 1e 06 cc e2 1c 84 b1 a3 5f 08 00 ..........._..
I don't know if the martian thing is relevant since i disabled rp_filter.
Thank you fLo
Update with more info
1) Yes, /proc/sys/net/ipv4/ip_forward
has been enabled.
2) enp0s3
which is the main host adapter in default network space has address of 10.0.2.15/24
3) Here is output of ip route
from default namespace
default via 10.0.2.2 dev enp0s3
10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15
172.20.0.0/24 dev br1 proto kernel scope link src 172.20.0.1
and from the testns
namespace
default via 172.20.0.1 dev veth-b
172.20.0.0/24 dev veth-b proto kernel scope link src 172.20.0.2
4) Here is ip a show
from host if useful.
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:3b:e4:70 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe3b:e470/64 scope link
valid_lft forever preferred_lft forever
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 4a:0a:90:fc:18:53 brd ff:ff:ff:ff:ff:ff
inet6 fe80::480a:90ff:fefc:1853/64 scope link
valid_lft forever preferred_lft forever
4: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:e4:cc:1e:06:cc brd ff:ff:ff:ff:ff:ff
inet 172.20.0.1/24 scope global br1
valid_lft forever preferred_lft forever
inet6 fe80::e4:ccff:fe1e:6cc/64 scope link
valid_lft forever preferred_lft forever
5: veth-a@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
link/ether 4a:0a:90:fc:18:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::480a:90ff:fefc:1853/64 scope link
valid_lft forever preferred_lft forever
6: veth-d@veth-c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
link/ether de:5d:d5:85:08:ee brd ff:ff:ff:ff:ff:ff
inet6 fe80::dc5d:d5ff:fe85:8ee/64 scope link
valid_lft forever preferred_lft forever
7: veth-c@veth-d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br1 state UP group default qlen 1000
link/ether 02:e4:cc:1e:06:cc brd ff:ff:ff:ff:ff:ff
inet6 fe80::e4:ccff:fe1e:6cc/64 scope link
valid_lft forever preferred_lft forever
>
I've updated the main post with answers, thanks. – Flo Woo – 2016-02-04T08:09:10.907