I'm trying to build up two redundant forward-facing load balancers (using either IPVS or NGINX), but first I'm trying to get the floating VRRP virtual-ip/mac to work properly before moving forward through the process.
I've got a stock Ubuntu 16.04 VM on VMware vSphere 6 with the latest updates. VM is on a DvSwitch portgroup with promiscuous mode, mac address change, and forged transmits enabled. VM is using VMXNET3 for NIC. I'm using the standard keepalived in the repo.
keepalived/xenial-updates,now 1:1.2.19-1ubuntu0.1 amd64 [installed]
I'm attempting to use the following configuration for keepalived...
root@lb1:~# cat /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface ens192
virtual_router_id 150
priority 150
use_vmac vrrp150
vmac_xmit_base
advert_int 1
virtual_ipaddress {
10.0.4.55
}
}
When initially set up, the pings toward the macvlan interface (vrrp150) are ARP replied with the parent interface MAC (which is bad for several reasons). I've attempted to use multitudes of different combinations of net.ipv4.conf settings laid out online, all of them seem to break ARP completely for the macvlan interface. iptables and ufw are completely disabled to my knowledge, and tcpdump shows the following...
root@lb1:~# tcpdump -s0 -i vrrp150
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vrrp150, link-type EN10MB (Ethernet), capture size 262144 bytes
10:05:19.271979 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:20.215301 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:21.215474 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:22.219300 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:23.215514 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:24.223971 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
10:05:25.224262 ARP, Request who-has 10.0.4.55 tell 10.0.4.31, length 46
So the ARP requests are indeed getting to the correct interface. I've attempted to use promiscuous mode on the physical interface, but it makes no difference (and the ARP requests get there regardless).
How can I get the keepalived virtual macvlan interface to respond to ARPs properly with the virtual MAC address so that traffic can be forwarded? I am trying to get the same virtual IP/MAC combination to follow between MASTER > BACKUP during transitions so that ARP tables are not affected on any upstream firewalls (GARP is not reliable/honored, and failover must be quick and as seamless as possible).