Netplan: Route multiple parallel internet connections with port forwarding to a single server with 1 network port

Question

Here's my network diagram:

        56.56.56.56 192.168.0.1/24
                    MAC:AA:BB:CC:DD:EE:01
               ___________     
            ---| Modem 1 |-------
            |  ___________      |             ___________
Internet ---|               | Switch |--------| Machine |
            |  ___________      |             ___________
            ---| Modem 2 |-------            192.168.0.3/24
               ___________     
        67.67.67.67 192.168.0.2/24
                    MAC:AA:BB:CC:DD:EE:02

Both modems forward the same ports from the internet to the machine.
The machine behind the modems should respond appropriately to any request coming from the internet. e.g. Modem 1 packets return via Modem 1, Modem 2 packets return via Modem 2.
The machine has only one network port and the switch is unmanaged.
The machine uses Netplan, iptables, and iproute2 for network configuration.

tudor -Reinstate Monica- · Accepted Answer · 2019-06-26T07:37:00.403

I eventually got my solution from article and comments in Policy Routing on Linux based on Sender MAC Address and the Netplan.io reference on Policy-Routing.

The trick is to mark and CONNTRACK incoming packets by source MAC address to a separate routing table via iptables -t mangle, and then tell Netplan to use the table to route outgoing packets appropriately.

First, we need tables for our packets to be herded into:

Append the following to the file /etc/iproute2/rt_tables:

1 modem1
2 modem2

Then, tell Netplan about the tables, routes, and marks:

network:
  version: 2
  ethernets:
    eth0:
      routes:
        - to: 0.0.0.0/0
          via: 192.168.0.1
          table: 1
        - to: 0.0.0.0/0
          via: 192.168.0.2
          table: 2
      routing-policy:
        - from: 0.0.0.0/0
          mark: 1
          table: 1
        - from: 0.0.0.0/0
          mark: 2
          table: 2

This first part tells netplan that packets in these different tables need different default routes. The second part says that some packets will have an fwmark from iptables, and these packets should be herded into those tables.

Then, tell iptables to mark packets by their originating MAC address, but only when it's not from the local network (a little script):

#!/bin/bash -x
MAC_MODEM1=AA:BB:CC:DD:EE:01
MAC_MODEM2=AA:BB:CC:DD:EE:02
MARK_MODEM1=0x1
MARK_MODEM2=0x2
LOCALNET=192.168.0.0/24

## Optional - Clear everything first
iptables -t mangle -F

for MODEM in MODEM1 MODEM2; do
    MAC=MAC_$MODEM
    MARK=MARK_$MODEM
    iptables --table mangle --append INPUT \
    --match state --state NEW \
    --match mac --mac-source ${!MAC} \
    ! --source $LOCALNET \
    --jump CONNMARK --set-mark ${!MARK}
done
iptables --table mangle --append OUTPUT \
--jump CONNMARK --restore-mark

Then, tell netplan to generate and apply:

$ sudo netplan generate
$ sudo netplan apply

e voila!

BONUS ANSWER

If you have more than one internal network (e.g. a VPN via a non-local IP), use ipset and iptables -m set ! -match-set alias, e.g.

ipset destroy officenets #optional - to clear

LOCALNET=192.168.0.0/24
VPNNET=10.10.10.0/29

ipset create privatenets hash:net
ipset add privatenets $LOCALNET
ipset add privatenets $VPNNET

then in the iptables script....

    iptables --table mangle --append INPUT \
    --match state --state NEW \
    --match mac --mac-source ${!MAC} \
    -m set \
    ! --match-set privatenets src \
    --jump CONNMARK --set-mark ${!MARK}

Verification

Verify fwmark rules to route tables:

$ ip rule
0:  from all lookup local 
0:  from all fwmark 0x1 lookup modem1 
0:  from all fwmark 0x2 lookup modem2 
32766:  from all lookup main 
32767:  from all lookup default

Verify iptables mangle routing:

$ sudo iptables -t mangle -L
...
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
CONNMARK   all  --  anywhere             anywhere             state NEW MAC AA:BB:CC:DD:EE:01 ! source 192.168.0.1/24 CONNMARK set 0x1
CONNMARK   all  --  anywhere             anywhere             state NEW MAC AA:BB:CC:DD:EE:02 ! source 192.168.0.2/24 CONNMARK set 0x2

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
CONNMARK   all  --  anywhere             anywhere             CONNMARK restore
...

Verify outgoing table routes:

$ ip route list table modem1
default via 192.168.0.1 dev eth0 proto static
$ ip route list table modem2
default via 192.168.0.2 dev eth0 proto static

Netplan: Route multiple parallel internet connections with port forwarding to a single server with 1 network port

1 Answers1