Why is virtualbox networking using virtio slow (bridged and host-only, Debian)

5

2

I'm currently installing a VirtualBox Server using Debian for Host and Client. I experience some trouble regarding network performance and CPU load especially using host-only networks and I've no idea how to track this further down.

Neither the host nor the client has a GUI available.

I ran several tests using iperf to get an idea what's going wrong.

Host = Virtualbox host (running iperf -s), Core i7 (4x2 cores @ 1,6 GHz), 16 GiB RAM

  • 192.168.0.13, eth0 (Broadcom Gigabit on-board), Gigabit Cat.6 via SoHo Gigabit Switch
  • 192.168.0.20, eth1 (Broadcom Gigabit on-board), Gigabit Cat.6 via SoHo Gigabit Switch
  • 192.168.14.1, vboxnet0, VirtualBox host-only network
  • OS: Debian 7.2.0 amd64 (Wheezy), Kernel 3.2.0-4-amd64, SMP Debian 3.2.51-1 x86_64
  • VirtualBox: 4.1.18_Debianr78361

Client = VirtualBox client(running iperf -s), 2 Cores, 2 GiB RAM

  • 192.168.0.14, eth0, bridged to host's eth1 using virtio
  • 192.168.14.100, eth1, vboxnet0, VirtualBox host-only network using virtio
  • lsmod on the virtual machine lists "virtio_net", "virtio_PCI", "virtio_ring" and "virtio"
  • installed virtualbox-guest-(dkms|utils|x11) and virtualbox-ose-guest-x11
  • OS: Debian 7.2.0 amd64 (Wheezy), Kernel 3.2.0-4-amd64, SMP Debian 3.2.51-1 x86_64

Other physical machine (running iperf -s)

  • 192.168.0.2, Gigabit Cat.6 via SoHo Gigabit Switch
  • OS: Ubuntu Server

Test 1: Loopback on physical machine performs great (>60 Gbit/s) and is bottlenecked by iperf-client which saturates one core:

  • Scenario: loopback on physical machine
  • Command: iperf -c 127.0.0.1 -B 127.0.0.1 -i 60 -t 600
  • Bandwidth(Mb/s): 62100 61900 61800 61900 61800 61900 61900 61900 61800 -> 61900
  • CPU: Host(iperf -s) 70 %, Host(iperf -c) 100 %

Test 2: Classical connection via eth0 works as expected:

  • Scenario: Host eth0 (Broadcom Gigabit on-board) -> Other physically wired machine via switch
  • Command: iperf -c 192.168.0.2 -B 192.168.0.13 -i 60 -t 600
  • Bandwidth(Mb/s): 942 941 941 941 941 941 941 941 941 -> 941
  • CPU: Host(iperf -c) 3.5 %

Test 3: Classical connection via eth1 works as expected:

  • Scenario: Host eth1 (Intel Gigabit Server 4xNIC PCIe) -> Other physically wired machine via switch
  • Command: iperf -c 192.168.0.2 -B 192.168.0.20 -i 60 -t 600
  • Bandwidth(Mb/s): 942 941 941 941 941 941 941 941 941 -> 941
  • CPU: Host(iperf -c) 3.5 %

Test 4: Loopback within virtual machine nearly saturates both virtual cores as expected; reaching 30 % of the host's native speed:

  • Scenario: Loopback entirely within virtual machine
  • Command: iperf -c 192.168.14.1 -B 192.168.14.100 -i 60 -t 600
  • Bandwidth(Mb/s): 19600 19500 19600 19500 19500 19600 19600 19500 19500 -> 19500
  • CPU: Host(VBoxHeadless) 200%, Client(iperf -s) 75 %, Client(iperf -c) 100 %

Test 5: Bridged virtio driver to outside world performs at 50 % (40 - 70 %) and occupies Host-CPU notably while client idles:

  • Scenario: Virtualbox eth0 virtio bridged to host eth1 -> Other physically wired machine via switch
  • Command: iperf -c 192.168.0.2 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s): 401 458 480 367 582 720 431 388 696 553 -> 508
  • CPU: Host(VBoxHeadless) 30-60 %, Client(iperf -c) 3 %

Test 6: Bridged virtio driver to host is bottlenecked by clients CPU, only reaching 12 % of the host's native speed!

  • Scenario: Virtualbox eth0 virtio bridged to host eth1 -> host's eth1
  • Command: iperf -c 192.168.0.20 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s): 7420 7660 7310 7620 7690 7580 7570 7580 7700 7710 -> 7580
  • CPU: Host(VBoxHeadless) 160 %, Host(iperf -s) 11 %, Client(iperf -c) 100 %

Test 7: Host-only networking using virtio is bottlenecked by clients CPU, only reaching 8 % of the host's native speed!

  • Scenario: Virtualbox vboxnet0 host-only virtio
  • Command: iperf -c 192.168.14.1 -B 192.168.14.100 -i 60 -t 600
  • Bandwidth(Mb/s): 4760 4740 4980 5300 4890 4560 5270 4850 5450 5070 -> 4990
  • CPU: Host(VBoxHeadless) 170 %, Host(iperf -s) 13 %, Client(iperf -c) 100 %

I added some more tests which led me to even more confusing results - something is definitely broken.

Test 8 = Test 6 using Intel 82545EM is slower than virtio

  • Scenario: Virtualbox eth0 Intel 82545EM bridged to host eth1 -> host's eth1
  • Command: iperf -c 192.168.0.20 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s): 3250 3300 3270 3290 3320 3380 3330 3330 3300 3300 -> 3310
  • CPU: Host(VBoxHeadless) 110 %, Host(iperf -s) 5 %, Client(iperf -c) 100 %

Test 9 = Test 6 using Intel 82543GC is very slow!

  • Scenario: Virtualbox eth0 Intel 82543GC bridged to host eth1 -> host's eth1
  • Command: iperf -c 192.168.0.20 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s): 667 627 717 743 633 653 621 677 701 623 -> 666
  • CPU: Host(VBoxHeadless) 35 - 117 %, Host(iperf -s) 5 - 17 %, Client(iperf -c) 50 - 100 %

Test 10 = Test 5 using Intel 82545EM -> fails to establish connection

  • Scenario: Virtualbox eth0 Intel 82545EM bridged to host eth1 -> Other physically wired machine via switch
  • Command: iperf -c 192.168.0.2 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s):
  • CPU: Host(VBoxHeadless) 30-60 %, Client(iperf -c) 3 %

Test 11 = Test 5 using Intel 82543GC, starts good, then decreases to a fraction!?

  • Scenario: Virtualbox eth0 Intel 82543GC bridged to host eth1 -> Other physically wired machine via switch
  • Command: iperf -c 192.168.0.2 -B 192.168.0.14 -i 60 -t 600
  • Bandwidth(Mb/s): 935 941 909 936 941 940 941 339 219 216 -> 732
  • CPU: Host(VBoxHeadless) 100 %, Client(iperf -c) 60 %

Q1: Why is the Loopback test (4) within the virtual machine three times slower than the same test on the host? Shouldn't it be near native speed?

Q2: Why does a common bridged network using virtio cause so much load on the host's side and doesn't reach 1 Gbps? Test (5)

Q3: Why is the bridge test (6) within the virtual machine eight times slower than the loopback test on the host (three times slower than loopback on virtual machine)?

Q4: Why is the host-only test (7) within the virtual machine twelve times slower than the loopback test on the host (four times slower than loopback on virtual machine)?

I expect some virtualization aspects to affect Q1-Q4. virtio seems to cause Q2-Q4 - but I have no idea where to investigate or to tweak.

All that doesn't sound too bad. But the problem originated in using the host-only network for iSCSI. It resulted in HDD-transfers as low as 10 MB/s and high CPU usage - the HDDs are capable of 147 MB/s. Something is definitely wrong ...

Kai Giebeler

Posted 2013-12-11T22:12:50.870

Reputation:

3Why are you expecting high performance? It's VirtualBox, after all; it's not like you're going to do anything important with it. – Michael Hampton – 2013-12-11T22:14:14.883

2What @MichaelHampton is hinting at is that VBox is a desktop virtualization solution, akin to VMware Workstation. You should not be running anything of importance on it. There are plenty of other high-performance options out there that should be used instead of Virtualbox. VMware, Hyper-V, XenServer, KVM, etc. – EEAA – 2013-12-11T22:16:12.567

1It's just for home use and I didn't expect that 1 Gbps is "high performance" - KVM is next on my list, if everything fails. – None – 2013-12-11T22:20:47.960

No answers

Asked: 2013-12-11T22:12:50.870

Viewed: 5 992 times

Active: 2013-12-12T17:31:46.467