4

There appear to be (at least) two ways to add a secondary IP address to an interface on Linux. By secondary, I mean that it'll accept traffic to the IP address, and responses to connections made to that IP will use it as a source, but any traffic the box originates (e.g., an outgoing TCP connection) will not use the secondary address.

Both ways start with adding the primary address, e.g., ip addr add 172.16.8.10/24 dev lan. Then I can add the secondary address with either a netmask of /24 (matching the primary) or /32.

If I add it with a /24, it gets flagged secondary, so will not be used as the source of outgoing packets, but that leaves a risk of the two addresses being added in the wrong order by mistake. If I add it with /32, wrong order can't happen, but it doesn't get flagged as secondary, and I'm not sure what the bad effects of that may be.

So, I'm wondering, which approach is least likely to break? (If it matters, the main service on this machine is MySQL, but it also runs NFSv3. I'm adding a second machine as a warm standby, and hope to switch between them by changing which owns the secondary IP.)

derobert
  • 1,288
  • 12
  • 22

2 Answers2

1

If I add it with a /24, it gets flagged secondary, so will not be used as the source of outgoing packets, but that leaves a risk of the two addresses being added in the wrong order by mistake. If I add it with /32, wrong order can't happen, but it doesn't get flagged as secondary, and I'm not sure what the bad effects of that may be.

I'm not sure what you mean here. You network interfaces are configured in a deterministic fashion when your system boots up. By properly configuring your networking you can decide which address is assigned first and which is assigned as the secondary address.

If you're on a RedHat-ish system, the primary network configuration file ifcfg-eth0 (for example) will be handled before any alias interfaces such as ifcfg-eth0:0. I'm less certain of the behavior on Debian-ish systems, but it looks like the /etc/network/interfaces file may be handled sequentially, so put your alias definitions after the primary address and you should be all set.

So, I'm wondering, which approach is least likely to break?

If both of your addresses are on the same network I don't know that either one is going to be more or less likely to break. For what it's worth, I usually configure secondary addresses with a /32 netmask, since the primary address already provides the necessary network route.

larsks
  • 41,276
  • 13
  • 117
  • 170
  • Debian based systems work the same using `eth0:1` etc in `/etc/network/interfaces`. On systems with lots of secondaries, I sometimes use the last octet of the ip address as the discriminator `10.0.0.9` -> `eth0:9`. I would try to use an existing failover solution for a case like this. – BillThor Jan 04 '11 at 00:04
  • Well, "wrong order by mistake" includes "administrator's mistake". The order IPs come up in from the config is deterministic enough (though things like parallel booting make it less so). `/etc/network/interfaces` is not sequential anymore, its normally the order udev adds the interfaces. – derobert Jan 04 '11 at 21:19
0

You cannot count on the addresses being "deterministic". In the case of failover configurations you may have something like pacemaker reaching in and removing or adding an IP to an interface. Since you don't know when pacemaker will install the IP you cannot guarantee order or rely on any such deterministic behavior.

I'm finding the connection tracking in linux a bit frustration because of this. I have a HA firewall configuration and outgoing packets for everything is sent on the fixed address. Thus, incoming traffic to the failover IP results in outgoing traffic from the fixed address (which is different) and thus doesn't appear established or related to the incoming connection and gets dropped.

WiegleyJ
  • 1
  • 1