Debian iptables Misconfiguration when Connected to VPN

0

1

TL;DR

Debian router works fine until I connect VPN, then some misconfiguration in iptables (or route tables) makes it so the router itself cannot ping IPs/Domains through tun0, therefore DNS servers. I need the router to ping outside IPs through tun0 (WAN), then it will be fully functional.

My Goal

So after trying stock router firmware, then trying DD-WRT, etc. I have come to the conclusion that I need a router running straight Debian on a custom built MicroATX PC, as I am using it as a VPN router for my entire network. I want the true autoconnect feature of a standard VPN linux app (NordVPN in this case), as any standard router firmware requires me to pick a server and I have to manually switch it if I want a different one.

What I've Done so Far

I have it successfully running as a router with the WAN and LAN interfaces setup properly (or at least functionally). I used DNSMasq with iptables and it runs great. Then I attempted to implement the VPN in my iptables (tun0 as WAN), and it sort of works.

The Problem

When I connect the VPN, and after running sudo service netfilter-persistent reload, I can ping IP addresses from a machine on the LAN interface perfectly fine. I cannot ping domain names, so I deduced it to be just a DNS problem. It's a bit deeper than that, since during debugging I found that my router cannot ping IP's/Domains outside of it's WAN's subnet when connected to the VPN. This would explain why I am unable to get DNS working on any machine on the LAN network, unless I manually set DNS on that machine (which makes it 100% functional).

My Question

What part of my iptables (or anything else) am I screwing up to make this happen? What do I need to add/remove to allow the router (being localhost) to ping the outside world so when it acts as the DNS server, client machines can fulfill requests properly?

iptables

Note that I want to allow routing with the VPN on or off, hence some duplicated rules (I don't fully understand iptables yet). Also note I have some untested port forwarding rules in place. Tips on those are welcome, but if I have issues with those I should probably make a separate question.

# WAN interface: eno1
# VPN interface: tun0
# LAN interfaces: enp3s0, enp4s0

*filter
# Allow all outgoing, but drop incoming and forwarding packets by default
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-I OUTPUT -o tun0 -j ACCEPT
-A OUTPUT -o eno1 -j ACCEPT

# Custom per-protocol chains
:UDP - [0:0]
:TCP - [0:0]
:ICMP - [0:0]

# Acceptable UDP traffic

# Acceptable TCP traffic

# Acceptable ICMP traffic

# Boilerplate acceptance policy
-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i lo -j ACCEPT

# Drop invalid packets
-A INPUT -m conntrack --ctstate INVALID -j DROP

# Pass traffic to protocol-specific chains
## Only allow new connections (established and related should already be handled)
## For TCP, additionally only allow new SYN packets since that is the only valid
## method for establishing a new TCP connection
-A INPUT -p udp -m conntrack --ctstate NEW -j UDP
-A INPUT -p tcp --syn -m conntrack --ctstate NEW -j TCP
-A INPUT -p icmp -m conntrack --ctstate NEW -j ICMP

# allow services (ssh, dhcp, dns)
-A INPUT -i enp3s0 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -i enp3s0 -p udp -m udp --dport 22 -j ACCEPT
-A INPUT -i enp4s0 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -i enp4s0 -p udp -m udp --dport 22 -j ACCEPT
-A INPUT -i eno1 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -i eno1 -p udp -m udp --dport 22 -j ACCEPT
-I INPUT -i enp3s0 -p udp -m udp --dport 67:68 -j ACCEPT
-I INPUT -i enp4s0 -p udp -m udp --dport 67:68 -j ACCEPT
-I INPUT -i enp3s0 -p tcp -m tcp --dport 53 -j ACCEPT
-I INPUT -i enp3s0 -p udp -m udp --dport 53 -j ACCEPT
-I INPUT -i enp4s0 -p tcp -m tcp --dport 53 -j ACCEPT
-I INPUT -i enp4s0 -p udp -m udp --dport 53 -j ACCEPT


# allow incoming traffic to the outgoing connections,
# et al for clients from the private network
-I FORWARD -i tun0 -o enp3s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i enp3s0 -o tun0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i tun0 -o enp4s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i enp4s0 -o tun0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eno1 -o enp3s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i enp3s0 -o eno1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eno1 -o enp4s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i enp4s0 -o eno1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# allow LAN to WAN
-I FORWARD -i enp3s0 -o tun0 -j ACCEPT
-I FORWARD -i enp4s0 -o tun0 -j ACCEPT
-A FORWARD -i enp3s0 -o eno1 -j ACCEPT
-A FORWARD -i enp4s0 -o eno1 -j ACCEPT

# accept initial connections (port forwarding)
-A FORWARD -i eno1 -o enp3s0 -p udp --dport 4134 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eno1 -o enp4s0 -p udp --dport 4134 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eno1 -o enp3s0 -p tcp --syn --dport 1022 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eno1 -o enp3s0 -p udp --dport 1022 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eno1 -o enp4s0 -p tcp --syn --dport 1022 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eno1 -o enp4s0 -p udp --dport 1022 -m conntrack --ctstate NEW -j ACCEPT

# allow subsequent traffic after initial connection (port forwarding)
-I FORWARD -i tun0 -o enp3s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i enp3s0 -o tun0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i tun0 -o enp4s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-I FORWARD -i enp4s0 -o tun0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eno1 -o enp3s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i enp3s0 -o eno1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eno1 -o enp4s0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i enp4s0 -o eno1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# prohibit everything else incoming
-I INPUT -i tun0 -j DROP
-A INPUT -i eno1 -j DROP

# Reject anything that's fallen through to this point
## Try to be protocol-specific w/ rejection message
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
COMMIT

*nat
-A PREROUTING -i eno1 -p udp -m udp --dport 4134 -j DNAT --to-destination 10.0.1.2:4134
-A PREROUTING -i eno1 -p tcp -m tcp --dport 1022 -j DNAT --to-destination 10.0.1.2:1022
-A PREROUTING -i eno1 -p udp -m udp --dport 1022 -j DNAT --to-destination 10.0.1.2:1022
-A POSTROUTING -o enp4s0 -p tcp --dport 1022 -d 10.0.1.2 -j SNAT --to-source 10.0.1.1
-A POSTROUTING -o enp4s0 -p tcp --dport 1022 -d 10.0.1.2 -j SNAT --to-source 10.0.1.1
-I POSTROUTING -o tun0 -j MASQUERADE
-A POSTROUTING -o eno1 -j MASQUERADE
COMMIT

*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

*security
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT

DNSMasq

I removed the army of comment lines in this for simplicity.

interface=enp3s0
interface=enp4s0

listen-address=127.0.0.1

dhcp-range=10.0.1.2,10.0.1.254,12h

Route Tables

Guide to interpret the following information:

[WAN=eno1, VPN=tun0, LAN=enp3s0 and enp4s0]
WAN Subnet: 10.0.0.0/24
LAN Subnet: 10.0.1.0/24
WAN IP for Debian router: 10.0.0.140
LAN IP for Debian router: 10.0.1.1

With VPN disconnected:

default via 10.0.0.1 dev eno1 
10.0.0.0/24 dev eno1 proto kernel scope link src 10.0.0.140 
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1 linkdown 
10.0.1.0/24 dev enp3s0 proto kernel scope link src 10.0.1.1 linkdown

With VPN connected:

0.0.0.0/1 via 10.8.3.1 dev tun0 
default via 10.0.0.1 dev eno1 
10.0.0.0/24 dev eno1 proto kernel scope link src 10.0.0.140 
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1 linkdown 
10.0.1.0/24 dev enp3s0 proto kernel scope link src 10.0.1.1 linkdown 
10.8.3.0/24 dev tun0 proto kernel scope link src 10.8.3.25 
128.0.0.0/1 via 10.8.3.1 dev tun0 
209.95.36.142 via 10.0.0.1 dev eno1

Traceroute

When the VPN is disconnected on the Debian router, tracerouting my gateway (so not the Debian router with iptables, but the router above it on its WAN interface eno1) produces:

traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
 1  box.local (10.0.0.1)  0.516 ms  0.686 ms  0.766 ms

But running it again with the VPN connected hangs for a bit, until finally producing this:

traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
 1  10.0.0.1 (10.0.0.1)  0.500 ms  0.785 ms  0.813 ms

So it looks like some (but not all) of the return information is getting dropped somehow.

Resources I used

I used this article, and this article, as well as some Google-Fu, to attempt setting up my iptables correctly.

Please note that tips/recommendations in the comments on how to improve stylization or remove redundancies are welcome. This is my first time working with iptables at all.

z7r1k3

Posted 2019-11-12T22:49:22.640

Reputation: 278

Since this question has yet to be answered, I'm beginning to wonder if the issue lies somewhere other than iptables... – z7r1k3 – 2019-11-17T00:03:17.437

I'm beginning to wonder if the issue lies somewhere other than iptables - Probably. Most VPN problems are routing problems first. Look at your roultes, look at traceroute output. Think about route tables on every system. Disable your iptables rules completely, except maybe your NAT rules. – Zoredache – 2019-11-20T21:03:05.450

@Zoredache The thing is, if I disable iptables, I can ping just fine over the VPN. It's the second I enable iptables that I get the problem. I haven't run a traceroute yet but I'll do that next. – z7r1k3 – 2019-11-21T02:20:58.307

@Zoredache so I did a traceroute, and get nothing. All asterisks. I tried going into iptables and allowing everything by default, input, output, forward, the works. After refreshing, that still didn't work. I'm beginning to wonder if this is a conflict between iptables and OpenVPN as connecting over the OpenVPN app gives the same result. – z7r1k3 – 2019-12-04T00:54:37.263

Question has been updated with a Route Tables section and a Traceroute section. – z7r1k3 – 2019-12-04T01:32:46.617

After reviewing the /help/on-topic for this site, I realize that this site is not for consumer networking, etc. Since this Router is deployed in a home environment, I have flagged it to be moved to Super User where it will be on-topic. – z7r1k3 – 2019-12-08T06:29:14.877

And to clarify, I am not deploying this as a business for a customer's home network. This is my personal router that I am deploying on my own home network, hence the decision to migrate the question. – z7r1k3 – 2019-12-08T06:36:47.573

Why don't you troubleshoot without all the iptables rules first? – Tom Yan – 2019-12-10T02:07:42.450

@TomYan I did. I erased all the rules and put allow on output, input, and forward traffic. And after an iptables restore and netfilter reload, same problem. Only when I disable iptables completely can I ping the outside world with the VPN connected. – z7r1k3 – 2019-12-11T00:50:20.897

I'm not sure if I need to make more changes than that to my iptables to get it to allow it through, or if I need to look into the route tables, or something else. I've spent hours googling and trying various things and nothing seems to have any effect on it. – z7r1k3 – 2019-12-11T00:55:41.200

It could be -I INPUT -i tun0 -j DROP (The -I makes it override -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT). It's not necessary anyway with the default policy of the INPUT chain being DROP. Your firewall rules are quite a mess btw. (IMHO if you have no idea what the rules mean or how iptables works, the firewall becomes pretty pointless.) – Tom Yan – 2019-12-11T01:27:33.420

@TomYan Well I'm kind of learning through this project, but the main point is to make it act as a router properly. And of course have the standard dropping policy of a router. But again, even with all the rules being erased except for input, output, and forward being set to ACCEPT, it still doesn't work. I found that out after the OP, so I think it might be a routing issue. I will try that though. – z7r1k3 – 2019-12-12T03:17:36.730

@TomYan And yeah I can tell they're a mess, I compiled them from about three different articles I copy/pasted. I just don't know it well enough to fix it. I probably know it better than I think at this point, but because it's not working don't trust myself to be honest. No real way for me to experiment and learn by breaking it until there's something working to break. – z7r1k3 – 2019-12-12T03:19:07.580

Let us continue this discussion in chat.

– Tom Yan – 2019-12-12T03:58:57.593

Answers

0

I've written rules that should fulfill your basic need:

*filter

-N CM_REJECT
-A CM_REJECT -p tcp -j REJECT --reject-with tcp-reset
-A CM_REJECT -p udp -j REJECT --reject-with icmp-port-unreachable
-A CM_REJECT -j REJECT --reject-with icmp-proto-unreachable

-N TCP
-A TCP -p tcp --syn -j ACCEPT
-A TCP -g CM_REJECT

-P FORWARD DROP

-N NEW

-N FWD_OUT
-A FWD_OUT -o tun0 -j ACCEPT
-A FWD_OUT -o eno1 -j ACCEPT

-N FWD_IN
-A FWD_IN -i enp3s0 -g FWD_OUT
-A FWD_IN -i enp4s0 -g FWD_OUT

-A NEW -s 10.0.1.0/24 -j FWD_IN

# rules for wan-to-lan port forwarding begin #

-N FWDPS
-A FWDPS -p udp --dport 4134 -d 10.0.1.2 -j ACCEPT
-A FWDPS -p udp --dport 1022 -d 10.0.1.2 -j ACCEPT
-A FWDPS -p tcp --dport 1022 -d 10.0.1.2 -g TCP

-N LAN
-A LAN -o enp3s0 -g FWDPS
-A LAN -o enp4s0 -g FWDPS

-A NEW -i eno1 -j LAN

# rules for wan-to-lan port forwarding end #

# Maybe? The -j in NEW (instead of -g) are for this btw.
#-A NEW -g CM_REJECT

-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m conntrack --ctstate NEW -g NEW

COMMIT

*nat

-N DNS_SERVER
-A DNS_SERVER -j DNAT --to-destination 8.8.8.8

-N DNS
-A DNS -p udp --dport 53 -g DNS_SERVER
-A DNS -p tcp --dport 53 -g DNS_SERVER

-A PREROUTING -d 10.0.1.1 -g DNS

# rules for wan-to-lan port forwarding begin #

-N FWDDST_A
-A FWDDST_A -j DNAT --to-destination 10.0.1.2

-N FWDPS
-A FWDPS -p udp --dport 4134 -g FWDDST_A
-A FWDPS -p udp --dport 1022 -g FWDDST_A
-A FWDPS -p tcp --dport 1022 -g FWDDST_A

-A PREROUTING -d 10.0.0.140 -g FWDPS

# rules for wan-to-lan port forwarding end #

-N MASQ
-A MASQ -o eno1 -j SNAT --to-source 10.0.0.140
-A MASQ -o tun0 -j MASQUERADE

-A POSTROUTING -s 10.0.1.0/24 -g MASQ

COMMIT

The rules in the filter table should allow traffics with source IP in the 10.0.1.0/24 range that comes in via either of the LAN interfaces to go out via either the WAN interface or the VPN tunnel. The replying traffics would be allowed as well. Other forwarding traffics will be dropped. (Note that the input and output traffics are not covered here, as they are not relevant to the "router" job.)

NAT (overload) are done (also for only traffics with source IP in the 10.0.1.0/24 range) in the nat table. SNAT is chosen for eno1 as the the WAN IP is assumed to be static. MASQUERADE is chosen for tun0 as the VPN IP is assumed to be dynamic.

I've also written rules that forward DNS requests (more precisely, traffics to tcp/udp port 53) that targeted the router to a real DNS server. You may or may not want/need them. (You can do it on DNS level with dnsmasq instead. Your configuration seems a bit off to me though.)

Update: rules that deals with the port forwarding you apparently need have been added; you don't need to do SNAT/MASQUERADE for those traffics unless this router (10.0.1.1) isn't the default gateway of the destination host(s). Also note that you probably need to do policy routing (fwmark) if you need the port forwarding to work when the the router is connected to the VPN, otherwise replies from the host(s) will be routed into the tunnel.

Side note: It appears to me that you would want to create a bridge and make enp3s0 and enp4s0 its ports. In that case, you should ditch the FWD_IN chain and the LAN chain (and make the rules that -g them -g the next chains, i.e. FWD_OUT and FWDPS respectively; -i/-o $bridge_name match should be added before the -g).

Update 2: The following rules are for the INPUT chain. Insert them before the first COMMIT or -P FORWARD DROP:

-P INPUT DROP

-N DNS
-A DNS -p udp --dport 53 -j ACCEPT
-A DNS -p tcp --dport 53 -g TCP

-N DHCP
# Shouldn't need 68 here
-A DHCP -p udp --dport 67 -j ACCEPT

-N SSH
-A SSH -p tcp --dport 22 -j ACCEPT

# This chain is intentionally created with -j rules instead of -g rules
-N MYJOBS
# Irrelevant to the requests that are being forwarded by the DNAT rules
#-A MYJOBS -j DNS
-A MYJOBS -j SSH
-A MYJOBS -j DHCP

-N MYPORTS
-A MYPORTS -i lo -j ACCEPT
-A MYPORTS -i eno1 -j SSH
-A MYPORTS -i enp3s0 -j MYJOBS
-A MYPORTS -i enp4s0 -j MYJOBS

# Maybe?
#-A MYPORTS -p icmp -j ACCEPT
#-A MYPORTS -g CM_REJECT

-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m conntrack --ctstate NEW -g MYPORTS

You can add -s and/or -d match to the rules in the MYPORTS chain in addition to the -i match btw. Not sure how much they matter though. And again, the enp{3,4}s0 rules can be replaced with a $bridge_name rule if they are made bridge ports of a same bridge.

Tom Yan

Posted 2019-11-12T22:49:22.640

Reputation: 4 744

Holy flip that's working! Thank you so much! (Sorry I took so long, school+holidays :P) I'll have to look into that fwmark option as I do need the port forwarding when the vpn is connected. fwmark is a bit over my head atm, so if you could point me to a good resource explaining how it's used, or include in a comment what I'd need so I can apply that that'd be appreciated :D I still have a ways to go before I'm able to write my own iptables apparently, but I'm learning everyday lol. – z7r1k3 – 2020-01-24T21:24:21.067

Kek, fun little bug, whenever I reboot, it will randomly select either enp3s0 or enp4so that will work properly. Only one works at a time, and it's random which one. I'm assuming I need to bridge, which I'll try next. Just after I google some info on it so I can try it lol. – z7r1k3 – 2020-01-25T00:42:16.553