I'm trying to add a Pacemaker virtual IP address; with it, the gateway, and the two VMs it serves on different subnets. I've only done this before with all IPs on the same subnet, and I need some help.
I have two VMs on 200.xx.xxx.9 and 200.xx.xxx.10 with the below in /etc/network/interfaces
auto eth0
iface eth0 inet static
address 200.xx.xxx.9
gateway 200.xx.xxx.9
netmask 255.255.255.255
post-up route add yy.yyy.yyy.1 dev eth0
post-up route add default gw yy.yyy.yyy.1
and
auto eth0
iface eth0 inet static
address 200.xx.xxx.10
gateway 200.xx.xxx.10
netmask 255.255.255.255
post-up route add yy.yyy.yyy.1 dev eth0
post-up route add default gw yy.yyy.yyy.1
They're both showing up in Pacemaker, and seemingly communicating OK. The bindnetaddr parameters in the two /etc/corosync/corosync.conf
files are:
bindnetaddr: address 200.xx.xxx.9
and
bindnetaddr: address 200.xx.xxx.10
respectively.
If everything was on the same subnet, I'd expect to add the virtual IP with something like:
sudo crm configure primitive eth0_virtual ocf:heartbeat:IPaddr params ip="200.zzz.z.162" nic="eth0" cidr_netmask="24" broadcast="200.zzz.z.255" op monitor interval="10s" timeout="20s"
and while this shows up as resource in crm_mon, it isn't allowing me to access one of the VMs via the virtual IP. Clearly there's more to it, but what?
Edit: After a suggestion on the ClusterLabs mailing list that this may be an underlying networking issue, here's where I'm at:
I'm hosting ESXi with online.net, who provide additional 'failover' IPs that can be assigned to my hosted VMs. I have three; 200.xx.xxx.9 and 200.xx.xxx.10 which I'm using for the two Ubuntu VMs listed, and the third, 200.zzz.z.162, which I'm trying to use as an ocf_heartbeat_IPaddr resource agent. Critically, online.net require that you match assign each IP a MAC address that matches an interface on a VM. I did this for the first two IPs (using different MAC addresses) but not for the third. When I tried using one of the existing MAC addresses with the third IP, I immediately got connectivity with:
So now, I have all three IPs and both VM interfaces sharing the same MAC address. This strikes me as a really bad idea, but, it actually seems to be working, without creating any packet errors or problems with connectivity or the HA cluster... at least none that I've spotted so far, other than martian traffic being logged. This is the ocf:heartbeat:IPaddr setting I'm using:
sudo crm configure primitive eth0_virtual ocf:heartbeat:IPaddr params ip="200.zzz.z.162" nic="eth0" cidr_netmask="32" broadcast="200.zzz.z.162" op monitor interval="10s" timeout="20s"