I found most of an answer here: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming
More info here: http://linux-ip.net/html/routing-tables.html
Also, thanks to Tom Yan, for providing an answer that had most of the needed parts. But I'm posting this answer because none of the others is really complete, totally accurate, or easy to follow.
The following process works for me on Centos 7. I could not get it to work with NetworkManager, because NetworkManager does not read /etc/sysconfig/network-scripts/rule-*
and route-*
, and I could not find an equivalent nmcli
command to make these changes persistent. If anyone has such a command, please share. So the first required step is to disable NetworkManager:
systemctl disable NetworkManager
systemctl enable network
systemctl stop NetworkManager
systemctl start network
Linux kernel 2.2 and 2.4 support multiple routing tables, each one numbered 0 to 255. The two routing tables normally employed are local
(routing table 255) and main
(routing table 254). These are listed in /etc/iproute2/rt_tables
. You can create a new routing table by choosing an unused number (look in /etc/iproute2/rt_tables
) and adding it to /etc/iproute2/rt_tables
.
In my case, the "primary" interface is eth0
, 192.168.20.20 with gateway 192.168.20.1, and the "mgmt" interface is eth1
, 192.168.5.5 with gateway 192.168.5.1.
I am choosing the new routing table number 200, and new routing table name mgmt
.
echo "200 mgmt" >> /etc/iproute2/rt_tables
Next, you need to create a rule to use the new routing table for traffic that was received on the mgmt interface. Other resources on the internet say you only need to do this for one interface, but you really need to do it on both, for the following reason: Notice, if you bring up only one interface, that interface is able to respond to ping, but if you bring up both interfaces, the one you brought up first responds to ping, and the second one doesn't. This means, if you bring up the mgmt
interface first and then the primary interface, the primary will not respond. So it's good practice to define the rules on both interfaces.
This can be done one-time (non-persistent) as follows:
ip rule add from 192.168.20.20 table main
ip route add default via 192.168.20.1 dev eth0 table main
ip rule add from 192.168.5.5 table mgmt
ip route add default via 192.168.5.1 dev eth1 table mgmt
At this point, both interfaces should be responding. Now, to make it persistent, notice, if you read /etc/sysconfig/network-scripts/ifup-routes
, it will parse all the rule-*
and route-*
files and pass each line as an argument to ip rule add
or ip route add
accordingly. So create four new files, as follows:
echo "from 192.168.20.20 table main" > /etc/sysconfig/network-scripts/rule-eth0
echo "default via 192.168.20.1 dev eth0 table main" > /etc/sysconfig/network-scripts/route-eth0
echo "from 192.168.5.5 table mgmt" > /etc/sysconfig/network-scripts/rule-eth1
echo "default via 192.168.5.1 dev eth1 table mgmt" > /etc/sysconfig/network-scripts/route-eth1
After reboot, you should find that both interfaces work by default.