1

I am trying to route traffic through a specific interface based on its destination.

OS is CentOS 7.

I am setting up a mini MPI farm where I have 2 slave workers and one master. The master and slaves use the built-in nic (em1) to communicate locally through a switch (this way all of the workers have "local" ips 192.168.1.*).

I then have a secondary nic (p2p1) that is connected to my router for external internet traffic.

As things are currently setup, I am able to ssh into each worker without problem but I cannot communicate with the internet (ping 8.8.8.8 states that it cannot access google's server).

From what I have searched for, I haven't been able to find a solution for this specific problem but I feel it is a simple fix...

typing route yields

Kernel IP routing table
Destination     Gateway      Genmask       Flags Metric Ref Use Iface
default         Linksys01356 0.0.0.0       UG    100    0     0 em1
default         gateway      0.0.0.0       UG    101    0     0 p2p1
10.11.230.0     0.0.0.0      255.255.255.0 U     101    0     0 p2p1
192.168.1.0     0.0.0.0      255.255.255.0 U     100    0     0 em1
192.168.122.0   0.0.0.0      255.255.255.0 U     0      0     0 virbr0

and typying iptables --list gives

Chain INPUT (policy ACCEPT)
target     prot opt source            destination
ACCEPT     udp  --  anywhere          anywhere            udp dpt:domain
ACCEPT     tcp  --  anywhere          anywhere            tcp dpt:domain
ACCEPT     udp  --  anywhere          anywhere            udp dpt:bootps
ACCEPT     tcp  --  anywhere          anywhere            tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source            destination
ACCEPT     all  --  anywhere          192.168.122.0/24    ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24  anywhere
ACCEPT     all  --  anywhere          anywhere
REJECT     all  --  anywhere          anywhere            reject-with icmp-port-unreachable
REJECT     all  --  anywhere          anywhere            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source            destination
ACCEPT     udp  --  anywhere          anywhere            udp dpt:bootpc

Ideally I would like all the MPI computers (master and slaves) to communicate using em1 and the master pull things from outside the local network on p2p1.

If it helps, the master IP is 10.11.230.43 for outside access and 192.168.1.143 on the local network, while one slave has the ip address 192.168.1.118.

If I am missing any other info please let me know.

Thank you

Matt Schramm
  • 13
  • 3
  • Doing more reading I found that you can list the interface using the -v command. In my iptables, all routes involve virbr0 which is associated with a 192.168.122.1 address while a virbr0-nic has no ip address. Both have a state of DOWN when looking at ip address. Do I then just need to add virbr0-nic into the ip tables? – Matt Schramm Nov 13 '18 at 22:17

1 Answers1

1

You have set a "default" route on both NICs, but "default" means, among other things, the number one. It is where traffic goes that has no other route. If you accidentally set more than one of them, then the lower metric decides which is used. In this case, that means your Internet bound traffic is going to "Linksys01356" on interface em1.

Because this isn't what you want, you should reconfigure the em1 interface to remove the gateway address. The only gateway that should be defined is the one which routes your traffic to the Internet.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • Thank you for the help, The second nic was simply placed onto the motherboard and the route was not added by me... but for clarification, I simply need to delete the links gateway entry? sudo route delete default gw Linksys01356 em1 Again thank you – Matt Schramm Nov 15 '18 at 14:47
  • You should also change the system's configuration (e.g. `/etc/sysconfig/network-scripts/ifcfg-em1`) to ensure that the other gateway is not configured there. – Michael Hampton Nov 15 '18 at 14:49