3

If I use SRIOV capable adapter for virtualization, and setup 2 different LACP links in 2 guests, so there may be 2 LACP links on the same ports on switch side, can switch handle this case?

For example:

switch port 0 <=> adapter port 0

PF(Host): eth0

VF1(Guest1-eth0): LACP team 1

VF2(Guest2-eth0): LACP team 2

===================

switch port 1 <=> adapter port 1

PF(Host): eth1

VF1(Guest1-eth1): LACP team 1

VF2(Guest2-eth1): LACP team 2

Thanks in advance.

jerome
  • 31
  • 1
  • 2

2 Answers2

3

This isn't going to work the way you expect it to. The switch does not know about PFs and VFs, it is just going to bond the two ethernet interfaces using LACP or not (the whole thing, not just traffic to one or another VF)

The way I have had to do this: Enable LACP on the PFs and bond them. Then you MUST also bond the VFs when they are passed to a guest. As jamieb mentioned, mac spoof checking is going to be a problem but there are ways around that. With Intel SR-IOV nics you can set an option that tells it to allow the VFs to set their own MACs and you can turn spoof checking off with "ip link". If you have something like the Broadcom NetExtremeII nics (bnx2 series of drivers) you can not turn off MAC spoofing or allow the guests to set their own MAC.

If using the Broadcoms you will need to do a little bit of pre-configuration before the guests can use them properly. After boot, enable the VFs, set their MAC addresses (they initially come up with the MAC as all zeros, that isn't usable), then you must disable and re-enable the VFs for those MAC addresses to actually be usable with the MAC addresses that you set. I assign them in pairs with the same mac, for example, port0-vf0 set mac to 22:27:1a:2a:d5:3a port1-vf0 set mac the same, pass both VFs to the guest (I use KVM) and in the qemu xml assign the MAC addresses the same as you set manually on the VFs. Then in the guest bond them using mode 2 and bonding will then work correctly.

With Intel, you just tell the PF that you want to turn off MAC snooping and allow the VF to set its own MAC address and then you can allow kvm to manage the MAC addresses and you don't need to do anything special with bonding, it will just work as if they are normal adapters.

The important thing is you run LACP ONLY on the PFs. Once that is done, the pairs of VFs from the PFs are basically already bonded at the switch end and MUST be bonded at the guest end for ALL vfs from those PFs. LACP is a link layer protocol, not a protocol by mac address. It doesn't bond paths to specific MAC addresses by LACP, it binds physical ports together and those physical ports carry the traffic for the PF and all of the VFs.

GeorgeB
  • 171
  • 3
  • Is it possible to use VFs (as bond) inside a VM and use the bond of the PFs as virutal NIC in another VM? I would like to still have the option to quickly move a VM to another Xen host when it doesn't need the performance/safety of a bonded VF. – Max Aug 13 '15 at 15:23
0

You'll need to disable spoof checking on your SR-IOV switches. When you bond the interfaces in the guests, the bond changes the interface MAC. An SR-IOV switch usually drops any frames which are not from its own PF or VFs. This is called spoof check.

Other than that, this should work fine.

suprjami
  • 3,476
  • 20
  • 29