2

I have a network with several Linux workstations/servers on it, and two Internet connections, each with a consumer-grade NAT router in front of it. So for concreteness, let's say the devices are:

192.0.2.0/26  network
192.0.2.1     router A
192.0.2.62    router B
192.0.2.10    host X
192.0.2.11    host Y

Here's a diagram, similar to one in the LARTC HOWTO:

                                                             ________
                                           +------------+        /
                                           |            |       |
         +-----------+-[NAT router A]------+ Provider 1 +-------
        _|                                 |            |     /
    ___/  \_         +--------------+      +------------+    |
  _/        \__      | Linux host   |                      /
 /             \     |              |                      |
| Local network -----+  if1         |                      |     Internet
 \_           __/    |              |                      |
   \__     __/       |              |                      \
      \___/          +--------------+     +------------+    |
        |                                 |            |     \
        +---------------[NAT router B]----+ Provider 2 +-------
                                          |            |       |
                                          +------------+        \________

If a new TCP connection comes in via one or the other router, return traffic needs to go out via that router. Otherwise the return SYN packet will be improperly NATted (if even passed out at all), and the connection will effectively be blackholed. I could set up mutually exclusive port-forwarding sets on the two routers if that would help, but I'd rather keep them the same (or at least in some cases overlapping) if possible.

For new outgoing connections, I want to allocate traffic by range using the main routing table, but for certain traffic I might want to randomly balance new connections across the two routers.

I suspect what I'm asking about is doable using iptables and ip, or perhaps either of those tools alone. However, the HOWTOs and other answers I've found seem to all address a single router (running Linux), or host, with multiple interfaces... not policy among multiple routers attached to the same interface.

Edit: I've found one other question, also unanswered, that's asking about the same situation (note 192.168.0.0/23 as the local network), but focuses on policy for outgoing traffic; my question here is specifically about policy for incoming traffic. (Both NAT devices have a "port forward by port-range to host" function.)

david
  • 263
  • 1
  • 11
  • http://www.lartc.org/lartc.html#LARTC.RPDB.MULTIPLE-LINKS – dmourati Aug 29 '16 at 22:33
  • @dmourati , that section of the HOWTO addresses one Linux host (acting as a router) with two interfaces directly to the respective networks, not a host choosing among two routers (attached via the same interface) according to some arbitrarily-complex policy that includes returning NAT traffic appropriately. – david Aug 30 '16 at 12:18
  • You should be able to cross-connect both upstreams to each router and then use an active-passive inbound to force all traffic through one router with something like keepalived. – dmourati Aug 30 '16 at 16:37
  • @dmourati, no, these are consumer-grade "router" devices in physically separate locations, each with only one physical uplink port (cable modem for one, DSL for the other). I'm not asking about what to do on or outside those devices, it is what it is. I'm asking about how to implement origin-sticky routing on a device attached to the local network. – david Aug 30 '16 at 16:58
  • Put a switch in the middle? – dmourati Sep 02 '16 at 05:01

1 Answers1

1

Yes, it's possible to do this using iptables and ip together. In my case all the Linux boxes at issue run Debian Stable, so I wrote a custom script for /etc/network/if-up.d.

# First set up two routing-tables
ip route add to default table 11 via 192.0.2.1  dev $IFACE
ip route add to default table 33 via 192.0.2.62 dev $IFACE

# ... and rules to use them.
ip rule add priority 99 table 11 fwmark 11
ip rule add priority 99 table 33 fwmark 33

# Copy connmark to fwmark for applicable outgoing packets.
iptables -t mangle -A OUTPUT ! -d 192.0.2.0/26 \
  -m addrtype --dst-type UNICAST -j CONNMARK --restore-mark

# Set connmark for applicable incoming packets.
iptables -A INPUT -i $IFACE ! -s 192.0.2.0/26 \
  -m addrtype --src-type UNICAST \
  -m mac --mac-source 00:00:5E:00:53:05 \
  -j CONNMARK --set-mark 11
iptables -A INPUT -i $IFACE ! -s 192.0.2.0/26 \
  -m addrtype --src-type UNICAST \
  -m mac --mac-source 00:00:5E:00:53:07 \
  -j CONNMARK --set-mark 33

Note that it's defined in that order so that the whole thing takes effect with the last two commands. 00:00:5E:00:53:05 and 00:00:5E:00:53:07 would be the on-link addresses of the two NAT devices.

david
  • 263
  • 1
  • 11