Testing network application with virtual network and artificial latency

Question

In order to test a network application in a controlled manner I want to create some sort of virtual network on a single machine in which I can control latencies. (I am running Ubuntu 14.04 x64). I have researched a bit and got up the following:

Using tap interfaces: vde_switch to create a virtual switch like so:

sudo vde_switch -tap tap0 -tap tap1
sudo ip addr add 10.0.0.100 dev tap0
sudo ip addr add 10.0.0.101 dev tap1

An alternative way to create the tap interfaces I've tried is:

sudo ip tuntap add dev tap0 mode tap  
sudo ip link set dev tap0 up  
sudo ip addr add 10.0.0.100 dev tap0  
[repeat for tap1 and any further interfaces]

So far so good. Now my problem is introducing artificial latency to these tap interfaces. I managed to do this using the following for actual interfaces like lo and eth0:

sudo tc qdisc add dev eth0 root netem delay 15ms

However, the above command does not seem to work with the tap interfaces. After I have executed it for one of the tap interfaces, nothing seems to have changed. The commend does not provide any output, so seems to find the interface alright and complete successfully. However, pinging e.g. 10.0.0.100 shows no change in latency.

Is there anything obvious I am missing? (Or am I approaching this completely in the wrong way?) Thank you!

Answer 1

I found a workaround in the end. It works fine in my situation but does require the use of a VM; there might well be a more elegant solution. Be warned.

This method uses VMware Workstation (note that Player is not sufficient). I used Ubuntu Desktop and Ubuntu Server as host and guest OS, respectively.

Workstation has a 'virtual network editor' (separate application packaged with Workstation). With that you can create additional virtual network interfaces and it turns out that you can assign virtual latencies to those independently. Create additional vmnetX interfaces as you need (mine were of type host-only) and you can add new network adaptors to your VM, manually specifying the corresponding network interfaces. If you're using linux you probably also need to add the new adaptors to /etc/network/interfaces in the guest OS.

Now you should have several different interfaces/networks available that are shared between host and guest. Introduce latency to those interfaces by using something like the following (google this command for more details, this is just one example):

sudo tc qdisc add dev vmnet1 root netem delay 20ms

I ended up introducing the delay in the guest OS (using the eth0, eth1, ... interfaces) instead of the host, as it is shown in the example above. I don't think it should make a massive difference, though.

Be aware that this is only a quick & dirty and not very generic description of how I got it to work. If you want to do something similar you might have to change a couple of details.

Answer 2

It's not pretty, but works:

#!/bin/bash

TAP0="tap0"
TAP1="tap1"
# use different subnets
IP0="10.0.31.1"
IP1="10.0.32.1"
# some fake ips in yet different subnets
FAKE0="10.1.31.1"
FAKE1="10.1.32.1"

ip tuntap add dev $TAP0 mode tap;
ip tuntap add dev $TAP1 mode tap;
ip addr add $IP0 dev $TAP0;
ip addr add $IP1 dev $TAP1;

ifconfig $TAP0 inet $IP0 netmask 255.255.255.0 up;
ifconfig $TAP1 inet $IP1 netmask 255.255.255.0 up;
ifconfig -a;

vde_switch -d -tap $TAP0 -tap $TAP1;

iptables -t nat -A POSTROUTING -s $IP0 -d $FAKE1 -j SNAT --to-source $FAKE0;
iptables -t nat -A PREROUTING -d $FAKE0 -j DNAT --to-destination $IP0;
iptables -t nat -A POSTROUTING -s $IP1 -d $FAKE0 -j SNAT --to-source $FAKE1;
iptables -t nat -A PREROUTING -d $FAKE1 -j DNAT --to-destination $IP1;

ip route add $FAKE1 dev $TAP0;
arp -i $TAP0 -s $FAKE1 $(ifconfig -a|grep $TAP1|awk '{print $5}');
ip route add $FAKE0 dev $TAP1;
arp -i $TAP1 -s $FAKE0 $(ifconfig -a|grep $TAP0|awk '{print $5}');

tc qdisc add dev $TAP0 root handle 1:0 netem delay 80ms 20ms 25% loss 5% duplicate 5%;
tc qdisc add dev $TAP1 root handle 1:0 netem delay 80ms 20ms 25% loss 5% duplicate 5%;

while this may be ugly as hell and probably plain wrong - it works as i need it to. note this only works between tap0 and tap1, not looping tap0 to tap0 or so. see the test-ping:

$ ping -qi 0.2 10.0.31.1 -I tap1                                                                                                                                                                          
PING 10.0.31.1 (10.0.31.1) from 10.0.32.1 tap1: 56(84) bytes of data.                                                                                                                                                            
^C                                                                                                                                                                                                                               
--- 10.0.31.1 ping statistics ---                                                                                                                                                                                                
313 packets transmitted, 301 received, +13 duplicates, 3% packet loss, time 62650ms                                                                                                                                              
rtt min/avg/max/mdev = 60.013/81.053/99.951/11.661 ms