5

I decided to play with veth : create a veth pair and send a ping from one end to another.

$ ip link add type veth
$ ip addr add 192.168.99.1 dev veth4
$ ip addr add 192.168.99.2 dev veth5
$ ip link dev veth4 set up
$ ip link dev veth5 set up

Let's check.

$ ip a

18: veth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 6a:dc:02:5b:f0:f3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.99.1/24 scope global veth4
       valid_lft forever preferred_lft forever
19: veth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 22:ec:d5:e8:7c:3e brd ff:ff:ff:ff:ff:ff
    inet 192.168.99.2/24 scope global veth5
       valid_lft forever preferred_lft forever

Everything seems to be ok. Now try to ping.

$ ping -I veth4 192.168.99.2
PING 192.168.99.2 (192.168.99.2) from 192.168.99.1 veth4: 56(84) bytes of data.
From 192.168.99.1 icmp_seq=1 Destination Host Unreachable
From 192.168.99.1 icmp_seq=2 Destination Host Unreachable
From 192.168.99.1 icmp_seq=3 Destination Host Unreachable


$ sudo tshark -i veth5
Capturing on 'veth5'
l  1   0.000000 6a:dc:02:5b:f0:f3 -> Broadcast    ARP 42 Who has 192.168.99.2?  Tell 192.168.99.1
1   2   1.003206 6a:dc:02:5b:f0:f3 -> Broadcast    ARP 42 Who has 192.168.99.2?  Tell 192.168.99.1

So veth5 recevies arp requests but doesn't bother to answer. What's the matter?

nshy
  • 151
  • 1
  • 4

3 Answers3

7

This is a very complex, very old question, involving several different areas, specifically VETH, Network Namespaces, ARP, route tables, and NAT...

I am adding the answer to this problem not just for OP, but for others out there, hopefully saving someone else from the grey hairs this gave me...

Suffice to say, after much research and testing -- veth pairs can exactly be used this way, as they are literally two seperate virtual interfaces. However, because they are virtual -- you will encounter some really odd problems with regards to ARP tables/entries. After several weeks of tackling this exact scenario (necessary for work), I have figured out exactly how to do this, and in the process have learned way more about ARP, NAT, and routing than I wanted to.

In order to accomplish exactly what you want, type the following (in my case, Ubuntu 16.04). Please note that once you are in the test namespace, you have to exit bash to leave, as everything (route tables, iptables, etc) from that point on is different than on the host machine. I usually open up two terminals, one that stays in the host -- the other that stays in the guest namespace.

Preface

# Tell the system we want to support IPv4 forwarding  
add/verify that the following is in `/etc/sysctl.conf`

`net.ipv4.ip_forward = 1`

# if it wasn't there, reparse syscrtl + restart networking
sysctl -p;
/etc/init.d/networking restart;

HOST SETUP

# create a veth pair
ip link add name vHOST type veth peer name vGUEST

# choose a private MAC address and private IP address
ifconfig vHOST hw ether 02:1d:8d:dd:0c:61
ifconfig vHOST 10.11.0.1/24 up

# have to setup routes   FROM HOST -> GUEST
ip route add 10.111.0.0/24 via 10.11.0.1 dev vHOST  # gateway

# have to explicitly assign what the MAC is for the vGUEST in the vHOST interface ARP table
arp -i vHOST -s 10.111.0.1 02:1d:8d:dd:0c:60 # vGUEST ip + mac address

# We must tell vHOST-vGUEST "tunnel", vHOST side that it's not a REAL bridge, so any ARP requests must be answered by vHOST
echo 1 > /proc/sys/net/ipv4/conf/vHOST/proxy_arp


# setup forwarding + NAT (so packets can come back)
iptables -A FORWARD -s 10.111.0.0/24 -o vHOST -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.111.0.1 -j SNAT --to 192.168.42.124

GUEST SETUP

# create the "test" namespace, so we can verfiy settings
ip netns add test

# add vGUEST "interface" to the "test" namespace
ip link set vGUEST netns test

# enter the "test" namespace with a bash shell
ip netns exec test bash

# choose a private MAC address and private IP address
ifconfig vGUEST hw ether 02:1d:8d:dd:0c:60
ifconfig vGUEST 10.111.0.1/24 up

# have to setup routes   FROM GUEST -> HOST
ip route add default via 10.111.0.1 dev vGUEST   # gateway

# have to explicitly assign what the MAC is for the vHOST in the vGUEST interface ARP table
arp -i vGUEST -s 10.11.0.1 02:1d:8d:dd:0c:61 # vHOST ip + mac address

now, GUEST/HOST can ping eachother, and vGUEST can ping outside the machine

Of paticular note, multimac can also do the exact same thing as veth, in a slightly different way, as multimac supports a many-to-one relationship, where veth is a one-to-one relationship.

In order to support this on without namespaces, you have to use policy routing (which is a bit more complex than network namespaces)

Nathan
  • 171
  • 1
  • 1
  • Unfortunately this renders veth interfaces unusable for me... Inserting arp entries manually is a hack that is not feasible for my use case. Isn't there any better solution for this? – Mike76 Nov 12 '18 at 17:52
  • I think that I might work around this limitation by creating a dummy interface and setting up a bridge between one of the veth interfaces and the dummy interface. I will try whether configuring a different ip address on the bridged dummy interface does the job. – Mike76 Nov 12 '18 at 18:48
4

Veth pairs aren't used this way. A veth pair is literally a device pipe, one end of the pipes packets come out the other end.

The simplest synonym I can offer is imagining one half of the pair is a ethernet device, whilst the other end is a switch port the device is plugged into. You shouldn't treat the pair as being two separate devices, but one device which has two 'ends' to push/pull from.

Matthew Ife
  • 22,927
  • 2
  • 54
  • 71
  • Well, I can use it this way for example if put one end in different namespace, I wonder why I could not do it without namespaces. – nshy Mar 29 '14 at 05:39
  • The same reason you cant do `ping -I eth0 ` when both are on the same network? When you dont specify the device (and it works) it actually uses `lo`. – Matthew Ife Mar 29 '14 at 07:04
  • Mmm, still don't understand. Hypothetical eth0 and eth1 are not connected. But veth0 and veth1 are. And when I put say veth1 in namespace and from host `ping -I veth0 veth1IP` - it works. – nshy Mar 29 '14 at 12:08
  • veth0 and veth1 are not connected, rather they are two ends to the same device, so it makes little sense to assign two addresses to it. My point about eth0 and eth1 is that if a test case without using a veth pair doesn't work, why would you expect a veth pair to work? – Matthew Ife Mar 29 '14 at 12:44
  • 1
    I have exactly the same misunderstanding as the OP. Why does it work with namespaces but not without? What is the difference? I do think of the veth as a pipe (well, as a "cable"), which can be used to connect two devices (say a host and a switch), so each device owns an end of the pipe, and does reply to ARP requests! (And in this scenario, it is common to have a different IP address assigned to each end of the pipe). – Norswap Nov 12 '14 at 17:37
0

try set these:

echo 1 > /proc/sys/net/ipv4/conf/all/accept_local
echo 1 > /proc/sys/net/ipv4/conf/default/accept_local
echo 1 > /proc/sys/net/ipv4/conf/veth4/accept_local
echo 1 > /proc/sys/net/ipv4/conf/veth5/accept_local
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/veth4/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/veth5/rp_filter