Ping not working from tap interface attached to bridge interface with ip on same subnet as that of bridge to outside

Question

My requirement is to create a multiple tap interfaces, each with ip address on same subnet.

I tried this by creating a bridge

               br0 (192.168.1.199)
     ___________|_____________________________________
     |                    |       |      |      |     |
   eth0                  tap0    tap1   tap2   tap3   tap4
                (192.168.1.150)  (.151) (.152) (.153) (.154)

I need all the tap interface be reachable from external PC. When i ping from tap0 to external computer say 192.168.1.200

ping -I tap0 192.168.1.200 -- the ping is not going through. 

But when i ping from 192.168.1.200 to 192.168.1.150 (tap0) it is working, but i get the mac address of the bridge (br0)

I have two problems:

1. How to ping from tap interface to external
2. How to get the mac address of the right tap interface, when pinged from outside.

Answer 1

This task seems like the XY problem, but I'll try to clarify, what is wrong with it and provide the solution.

1. When you assign the ip address on the interface, the kernel automatically creates the directly-connected route in the main routing table (what you see in the ip route list) and special routes (local and broadcast in the local table; what you see in the ip route list table local).

2. When you assign the overlapped or the same address spaces on several interfaces, you will get the several identical routes in the routing tables. Obviously, this isn't a good thing.

3. At result the select of particular route depends on address assignment order and some other factor. To check it you can use the ip route get 192.168.1.200 command. When you will try recreate the bridge interface, likely you will lost the connectivity completely, because the route to 192.168.1.0/24 subnet will point through one of tap interfaces.

4. This configuration will works only in one case: when the address on the bridge interface is assigned before, than on other interfaces with same subnet, but there isn't a guarantee. So, the main result: don't assign the addresses from same subnet on several interfaces if you're not complete understand how it will work.

5. More correct way to make this configuration to work is assignment of addresseses on the tap interfaces with /32 prefix length. It works as expected in most part of time.

6. In the linux kernel, the ip addresses are not hard linked with interface. Small demonstration: if your host have two NIC with various address and you physically disconnect cable from one of them, you will able ping to ip address on the disconnected NIC through working NIC from outside.

7. Selection of interface for communication with external hosts is defined by routing configuration when you specify an ip address in the -I option of the ping. It just the specify the source address of outgoing packets, but not specify the output interface. The mac address is filled at last steps of sending of the packets. You can check the actual route with ip route get command:

gw:~# ip r ls table local
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev br0 proto kernel scope link src 192.168.1.199 
local 192.168.1.150 dev tap0 proto kernel scope host src 192.168.1.150 
local 192.168.1.151 dev tap1 proto kernel scope host src 192.168.1.151 
local 192.168.1.152 dev tap2 proto kernel scope host src 192.168.1.152 
local 192.168.1.153 dev tap3 proto kernel scope host src 192.168.1.153 
local 192.168.1.154 dev tap4 proto kernel scope host src 192.168.1.154 
local 192.168.1.199 dev br0 proto kernel scope host src 192.168.1.199 
broadcast 192.168.1.255 dev br0 proto kernel scope link src 192.168.1.199 

gw:~# ip -4 r ls
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.199 

gw:~# ip r g 192.168.1.200
192.168.1.200 dev br0 src 192.168.1.199 uid 0 
    cache 

gw:~# ip r g 192.168.1.200 from 192.168.1.150
192.168.1.200 from 192.168.1.150 dev br0 uid 0 
    cache

8. But when you specify an interface name in the -I option of the ping, the raw socket will be used. In this case the output interface is hard defined and the routing step is skipped. But, as described in other answer, the kernel will attempt to send the packet through specified interface (inactive tap). Obviously, it fails.

9. So we have the good example of the XY problem. I think the topic starter wanted to test the bridging with emulating of separate hosts. One of simple ways to do it is usage of network namespaces.

Small example

Brief example for ip address 192.168.1.150.

• Create the network namespace, that will emulate the virtual host:

ip netns add VM150

• Create the veth interfaces pair. Move out the peer interface into the network namespace. The remained veth interface attach to the br0 interface as a bridge port.

ip link add name vethVM150 type veth peer vethIntVM150
ip link set dev vethIntVM150 netns VM150
ip link set dev vethVM150 master br0
ip link set dev vethVM150 up

• Configure the network inside the VM150 network namespace. And then ping the 192.168.1.200 host from it:

ip netns exec VM150 bash
ip netns identify
output> VM150
ip link set up dev lo
ip link set up dev vethIntVM150
ip address add 192.168.1.150/24 dev vethIntVM150
ping 192.168.1.200
...

• Verify the configuration with the tcpdump and bridge utils (run in other console of host; better use the GNU screen or the tmux). You will see the different mac address in the ethernet headers.

tcpdump -ni eth0 -e
15:38:54.084416 0a:28:39:f0:04:ad > 0c:d6:26:25:f9:00, ethertype IPv4 (0x0800), length 98: 192.168.1.150 > 192.168.1.200: ICMP echo request, id 2037, seq 72, length 64
15:38:54.088262 0c:d6:26:25:f9:00 > 0a:28:39:f0:04:ad, ethertype IPv4 (0x0800), length 98: 192.168.1.200 > 192.168.1.150: ICMP echo reply, id 2037, seq 72, length 64
15:38:54.403666 0c:d6:26:c2:70:00 > 0c:d6:26:25:f9:00, ethertype IPv4 (0x0800), length 98: 192.168.1.199 > 192.168.1.200: ICMP echo request, id 2043, seq 15, length 64
15:38:54.407580 0c:d6:26:25:f9:00 > 0c:d6:26:c2:70:00, ethertype IPv4 (0x0800), length 98: 192.168.1.200 > 192.168.1.199: ICMP echo reply, id 2043, seq 15, length 64
15:38:55.085501 0a:28:39:f0:04:ad > 0c:d6:26:25:f9:00, ethertype IPv4 (0x0800), length 98: 192.168.1.150 > 192.168.1.200: ICMP echo request, id 2037, seq 73, length 64
15:38:55.087252 0c:d6:26:25:f9:00 > 0a:28:39:f0:04:ad, ethertype IPv4 (0x0800), length 98: 192.168.1.200 > 192.168.1.150: ICMP echo reply, id 2037, seq 73, length 64
15:38:55.405129 0c:d6:26:c2:70:00 > 0c:d6:26:25:f9:00, ethertype IPv4 (0x0800), length 98: 192.168.1.199 > 192.168.1.200: ICMP echo request, id 2043, seq 16, length 64
15:38:55.407533 0c:d6:26:25:f9:00 > 0c:d6:26:c2:70:00, ethertype IPv4 (0x0800), length 98: 192.168.1.200 > 192.168.1.199: ICMP echo reply, id 2043, seq 16, length 64
15:38:56.087472 0a:28:39:f0:04:ad > 0c:d6:26:25:f9:00, ethertype IPv4 (0x0800), length 98: 192.168.1.150 > 192.168.1.200: ICMP echo request, id 2037, seq 74, length 64
15:38:56.091242 0c:d6:26:25:f9:00 > 0a:28:39:f0:04:ad, ethertype IPv4 (0x0800), length 98: 192.168.1.200 > 192.168.1.150: ICMP echo reply, id 2037, seq 74, length 64

• Verification on the 192.168.1.200 host is pretty simple - enough check the ARP table

host:~# ip n ls dev eth0
192.168.1.150 lladdr 0a:28:39:f0:04:ad REACHABLE
192.168.1.199 lladdr 0c:d6:26:c2:70:00 REACHABLE

Answer 2

Re question 1:

ping -I tap0

tells ping to send the ping packet out on tap0. This will bypass the bridge and really only send on the specified "physical" interface. So, effectively, you're not pinging "from" the tap interface, you're pinging "to" it.

If you want to ping "from" the tap interface, you need to attach something to it (e.g. OpenVPN) and send the ping from the other end of the virtual cable the tap interface is connected to.

Re question 2: You could try something like

arp -i br0 -Ds 192.168.1.150 tap0 pub

etc.