How can I do IP Passthrough on a Linux box?

Question

I have a Raspberry Pi, running Raspbian (Debian Based), with 2 network interfaces.

One of the interfaces is the WAN, coming from a Satellite Modem. The other is the Customer Router, which the Pi is providing the internet for.

The problem is that I'm only dealing with a /30, and the Satellite Modem already has one of the two IPs on that /30, so I only have one IP left (1.1.1.2/30), and 2 devices that want it (my Raspberry Pi, and the Customer Router).

I know I can easily do NAT, and give the customer a LAN IP. I don't want that. I need the customer to have a public IP. I also know that I can configure the Pi to just be a switch, with no IP of its own, but then the Pi won't have internet access. I don't want that either. I need both the Pi, and the customer router to have internet.

I know this is possible, because I've used MicroHard cell modems in the past which do this exact thing. The Cell Modem gets a public IP when it connects to the cell tower, and when you enable IP passthrough mode on it, a router plugged into that cell modem will also get the same public IP.

If you hit that public IP from the internet on port 80, 443, or 22, you'll be talking to the cell modem, but if you hit any other port, you'll be talking to the router behind the cell modem. That's exactly what I want to do. I'm just not sure exactly how they do it, and how I replicate it in Linux. MicroHard Cell Modems are linux-based, so it's possible on Linux.

So far, this is my plan:

• Static IP of 1.1.1.2/30 on Raspberry Pi's eth0 (Sat Modem Interface)
• Static IP of 1.1.1.254/24 on Raspberry Pi's eth1 (Customer Router Interface)
• DHCP server on eth1 giving the Customer Router:
  • 1.1.1.2/24
  • DG: 1.1.1.254
• Enabled net.ipv4.ip_forward
• Because the 2 subnets on the 2 interfaces obviously overlap, use policy based routing rules to tell the Pi when and where it should send traffic

It now looks like this:

I've been trying to get this working, but that last bullet point is obviously the hard part. Everything else (DHCP Router, Static IPs, IP Forwarding) is all done, and working.

I've done this:

# Rules to prevent ARP from going out the wrong interface
$ sudo arptables -A INPUT -i eth0 --destination-ip ! 1.1.1.2 -j DROP
$ sudo arptables -A INPUT -i eth1 --destination-ip ! 1.1.1.254 -j DROP

# Create custom routing tables
$ sudo vim /etc/iproute2/rt_tables
    190 to_sat_modem
    200 to_customer_router

# Add rules to tables
$ sudo ip route add default dev eth0 table to_sat_modem
$ sudo ip route add default dev eth1 table to_customer_router

# Add policy based routing rules
$ sudo ip rule add fwmark 0x1 lookup to_sat_modem
$ sudo ip rule add fwmark 0x2 lookup to_customer_router

# Add masq rule
$ sudo iptables -t nat -A POSTROUTING -o eth0 ! -s 1.1.1.2 -j SNAT --to 1.1.1.2

# Add rules for when to use these routing tables

$ sudo iptables -t mangle -N TO_CUSTOMER_ROUTER
$ sudo iptables -t mangle -N TO_SAT_MODEM

$ sudo iptables -t mangle -A OUTPUT -d 1.1.1.2 -j TO_CUSTOMER_ROUTER        # From localhost
$ sudo iptables -t mangle -A PREROUTING -i eth0 -d 1.1.1.2 -p udp -j TO_CUSTOMER_ROUTER                                         # All UDP ports
$ sudo iptables -t mangle -A PREROUTING -i eth0 -d 1.1.1.2 -p tcp --match multiport ! --dports 22,80 -j TO_CUSTOMER_ROUTER  # Most TCP ports
$ sudo iptables -t mangle -A TO_CUSTOMER_ROUTER -j MARK --set-xmark 0x2
$ sudo iptables -t mangle -A TO_CUSTOMER_ROUTER -j ACCEPT

$ sudo iptables -t mangle -A OUTPUT -d 1.1.1.1 -j TO_SAT_MODEM      # From localhost
$ sudo iptables -t mangle -A PREROUTING -i eth1 -d 1.1.1.1 -j TO_SAT_MODEM
$ sudo iptables -t mangle -A TO_SAT_MODEM -j MARK --set-xmark 0x1
$ sudo iptables -t mangle -A TO_SAT_MODEM -j ACCEPT

It feels like I'm on the right track, but I can't seem to get it to work. Any ideas?

PS. Yes, I know that most of my rules won't survive a reboot. I'll fix that once I have a working solution

PPS. This is trying to achieve one-way transparency. The Sat Modem will only see a single device behind it (1.1.1.2), while the Customer Router will be able to talk to the Pi (1.1.1.254) and the Sat modem (1.1.1.1). This should work, but has the problem of masking an entire /24 of real public IPs. I can change the /24 to a /29 to reduce the effect, but it'll still happen. Eventually, I'll want 2-way transparency so that I can just reuse the /30 IPs on both ends of the Pi, but I figured this would be easier to start with

PPPS. I'm aware that the obvious answer to this question is

Don't do this. Do something else. Give the customer a LAN IP, or put your Pi behind the customer router and give it a LAN IP, or put the pi in bridge mode, or ...

I don't want to write a 10 page explanation for why this needs to be done this way, but trust me, it needs to be done this way, if at all possible, which, like I said, I'm pretty sure is possible because I've seen it done

Answer 1

Based on my 2 Pi Solution, I was able to use Linux network namespaces to configure the same thing in a single Pi:

Initial Steps:

• Enable IP forwarding
• Set your eth0 to 1.1.1.2/30 static IP as you normally would
  • You could use a DHCP client, but you'd need some sort of hook that reconfigures the network for the new IPs

Script to configure networking:

#!/bin/bash

###############
# Global Vars #
###############

# Sat modem IP with subnet
# Ex: 1.1.1.1
SAT_MODEM_IP="1.1.1.1"

# Router IP
# Ex: 1.1.1.2
ROUTER_IP="1.1.1.2"

# Subnet in cidr notation
# Ex: 30
SUBNET_MASK="30"

# Space separated list of ports for the Pi to intercept
PORTS="8081 8082"

#####################
# Calculated Values #
#####################

# Removes the slash, in case the user wrote "/30" instead of "30" above
SUBNET_MASK=$(echo $SUBNET_MASK | tr -d '/')

# Make sure port list is property formatted for iptables
PORTS=$(echo $PORTS | sed 's/ /,/g' | tr -s ',')

##########
# Checks #
##########

# Make sure we are running as root
if ! [ $(id -u) = 0 ]
then
    echo "Error: This script must be run as root" >&2
    exit 1
fi

# Make sure ns2 doesn't already exist
if /sbin/ip netns | grep -q '^ns2'
then
    echo "Error: ns2 already exists"
    exit 1
fi

###################
# Start of Config #
###################

# Create ns2 namespace
ip netns add ns2 

# Move eth1 to ns2 namespace
ip link set eth1 netns ns2

# Configure eth1 in ns2 namespace
ip netns exec ns2 ip address add $SAT_MODEM_IP/$SUBNET_MASK dev eth1
ip netns exec ns2 ip link set eth1 up

# Enable loopback interface in ns2 namespace
ip netns exec ns2 ip link set lo up

# Create virtual interface pair, and move one of the interfaces into
# the ns2 namespace
ip link add veth type veth peer name vpeer
ip link set vpeer netns ns2

# Set ip on virtual interfaces
ip address add 192.168.90.5/24 dev veth
ip netns exec ns2 ip address add 192.168.90.6/24 dev vpeer

# Bring up both virtual interfaces
ip link set veth up
ip netns exec ns2 ip link set vpeer up

# Add DG to ns2 namespace
ip netns exec ns2 ip route add default via 192.168.90.5

# Add root namespace iptables rules
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o veth -s $SAT_MODEM_IP -j MASQUERADE
iptables -t nat -A PREROUTING -i eth0 -p udp -j DNAT --to-destination 192.168.90.6
iptables -t nat -A PREROUTING -i eth0 -p tcp -m multiport ! --dport $PORTS -j DNAT --to-destination 192.168.90.6
iptables -t nat -A PREROUTING -i eth0 -p ICMP -j DNAT --to-destination 192.168.90.6
iptables -t nat -A PREROUTING -i veth -p udp -d 192.168.90.5 -j DNAT --to-destination $SAT_MODEM_IP
iptables -t nat -A PREROUTING -i veth -p tcp -m multiport ! --dport $PORTS -d 192.168.90.5 -j DNAT --to-destination $SAT_MODEM_IP
iptables -t nat -A PREROUTING -i veth -p ICMP -d 192.168.90.5 -j DNAT --to-destination $SAT_MODEM_IP

# Add ns2 namespace iptables rules
ip netns exec ns2 iptables -t nat -A POSTROUTING -o vpeer -j MASQUERADE
ip netns exec ns2 iptables -t nat -A POSTROUTING -o eth1 -s 192.168.90.5 -j MASQUERADE
ip netns exec ns2 iptables -t nat -A PREROUTING -i vpeer -j DNAT --to-destination $ROUTER_IP
ip netns exec ns2 iptables -t nat -A PREROUTING -i eth1 -d $SAT_MODEM_IP -j DNAT --to-destination 192.168.90.5

# Add ICMP blocking
# Note: This just prevents customers from seeing LAN IPs in a
# traceroute from site, and duplicate WAN IPs in a traceroute to
# site
iptables -A OUTPUT -p ICMP -j DROP
ip netns exec ns2 iptables -A OUTPUT -p ICMP -j DROP

Note: I've never used network namespaces before, but they ended up being super easy to understand, and use. See: https://blogs.igalia.com/dpino/2016/04/10/network-namespaces/

Also note: You could setup a DHCP server on eth1 to give the customer router an IP without them having to statically set one, but you'll need to run your DHCP server on eth1 in the ns2 namespace. That is outside the scope of this question, but look for the NetworkNamespacePath systemd option, and make sure you know how to properly edit a systemd unit file without fighting with your package manager.

Answer 2

In the Question, I started going down the path of trying to do 1-way transparency, because I thought that would be an easier place to start, despite the fact that my ultimate goal was 2 way transparency.

By one way transparency, I mean that the modem can only see a single device behind it (1.1.1.2), but the Customer Router can see both the Pi (at 1.1.1.254) and the modem at 1.1.1.1.

The problem with 1-way transparency, at least the way I was trying to do it, is that I would be masking legitimate public IPs. This is obviously not ideal.

The following is one way to get 2-way transparency working, but it uses 2 Raspberry Pis. While this works, and may help someone else, I need this exact thing working on a single Pi, so I'll be looking into using Linux Network Namespaces to get this done on a single Pi. If anyone has a working solution on a single Pi, let me know.

2 Pi Solution:

Generic Steps:

• Configure Static IPs on both Pis
• Enable routing on both Pis (echo 1 > /proc/sys/net/ipv4/ip_forward)

Pi 1:

ip route add default via 1.1.1.1

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth1 -s 1.1.1.1 -j MASQUERADE

# All UDP Ports
iptables -t nat -A PREROUTING -i eth0 -p udp -j DNAT --to-destination 192.168.90.6

# Some TCP Ports
iptables -t nat -A PREROUTING -i eth0 -p tcp -m multiport ! --dport 8081,8082 -j DNAT --to-destination 192.168.90.6

Pi 2:

ip route add default via 192.168.90.5

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth1 -s 192.168.90.5 -j MASQUERADE

# Forwards EVERYTHING to Customer Router
iptables -t nat -A PREROUTING -i eth0 -j DNAT --to-destination 192.168.90.6

Note: I moved on from this 2-pi solution, to a single-pi solution fairly quickly, so this isn't as worked out as the accepted single-pi solution, and could use some improvements if anyone plans to use this. This is mostly here only for reference, as it is easier to understand than the single-pi solution, but essentially does the same thing