libvirt KVM virtualisation - define some guests to use one NIC, others to use a different NIC?

Question

I am running 30 libvirt-defined KVM VMs on a server with two physical NICs (1Gbps).

Currently I'm using NATted virtual networks, defining the VMs' interfaces (as part of a full VM definition, not a separate network definition) like this:

<interface type='bridge'>
  <source bridge='virbr0'/>
  <target dev='vnet0'/>
  <model type='virtio'/>
  <alias name='net0'/>
</interface>

[EDIT] This works as intended. The 30 VMs use the host's default ethernet interface - but the bandwidth on this caps out at 1Gbps.

To maximise throughput (specifically RX) for the VMs, I'd like half of the VMs to use one of the physical NICs, and the other half to use the other NIC. The host will also have to use one of these NICs.

Is it possible to do this? I'm looking for maximum RX performance on the VMs.

We can't use bridged mode, as we likely don't have enough IPs available for the number of VMs we intend to run (on multiple copies of this server).

Answer 1

Earlier this year I had the same problem and this is what I came up with after some research.

In general, the logistics are the following; since you have the xml files created, you need to first define the network, start and enable it to autorun with the start of the service and then you need to attach it to the VM of your choice.

For example, if you have a net0.xml file then you execute:

$ sudo virsh net-define net0.xml   //To define the network from the xml file without starting it
$ sudo virsh net-start net0    //To start an already defined network 
$ sudo virsh net-autostart net0 //To configure the net0 network to automatically start along with the service

To confirm the creation of the bridge you can issue:

$ ip a 

or

$ virsh net-list --all 

Now to attach a network to a VM you execute:

$ sudo virsh attach-interface --domain <vm-name-here> --type bridge \
--source net0 --model virtio --config --live 

With the following command you can check what virtual networks you have attached to a specific VM.

$ sudo virsh domiflist <vm-name-here>

Interface Type Source Model MAC
-------------------------------------------------------
vnet0 bridge virbr0 virtio 52:54:00:e9:ad:17
vnet1 bridge net0 virtio 52:54:00:47:e1:eb

If you need to detach a network from your VM (for example the default network or one previously created and attached) then you can execute the above command to copy the MAC address, and then the following to detach the interface from your VM.

$ sudo virsh detach-interface --domain <vm-name-here> --type bridge --mac
52:54:00:47:e1:eb --config

$ sudo virsh domiflist <vm-name-here>
Interface Type Source Model MAC
-------------------------------------------------------
vnet0 bridge net0 virtio 52:54:00:e9:ad:17

If for any reason the bridge is not sonnected you can issue:

$ sudo nmcli connection up net0

And to see the specifics of the net0 connection you can do so with:

$ sudo nmcli connection show net0

With the following command you can check the network list you have created for your VMs and also you can check the flags that correspond to the state, persistence and autostart of each network.

virsh net-list --all 

Everything is documented here and they say it better than I do.

Generally, i have found many answers in the following sites, regarding either the KVM hypervisor or linux administration:

tecmint

RedHat-Network interfaces documentation

Also, maybe you'd like to check Cockpit. It provides a web-based interface for your server and you can add it to your machine via the regular repository. With the addition of the packages cockpit-machines you can have a pretty neat management environment for your server. I recommend that you install it manually though, as the version in the repository is a few versions back from the current.

Hope this helps.