32

I have a VPS running CentOS 7 that I connect to with SSH. I would like to run an OpenVPN client on the VPS so that internet traffic is routed through the VPN, but still allow me to connect to the server via SSH. When I start up OpenVPN, my SSH session gets disconnected and I can no longer connect to my VPS. How can I configure the VPS to allow incoming SSH (port 22) connections to be open on the VPS's actual IP (104.167.102.77), but still route outgoing traffic (like from a web browser on the VPS) through the VPN?

The OpenVPN service I use is PrivateInternetAccess, and an example config.ovpn file is:

client
dev tun
proto udp
remote nl.privateinternetaccess.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
tls-client
remote-cert-tls server
auth-user-pass
comp-lzo
verb 1
reneg-sec 0
crl-verify crl.pem

VPS's ip addr:

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens33:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:be:16:f7 brd ff:ff:ff:ff:ff:ff
    inet 104.167.102.77/24 brd 104.167.102.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:febe:16f7/64 scope link
       valid_lft forever preferred_lft forever
4: tun0:  mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 100
    link/none
    inet 10.172.1.6 peer 10.172.1.5/32 scope global tun0
       valid_lft forever preferred_lft forever

VPS's ip route:

0.0.0.0/1 via 10.172.1.5 dev tun0
default via 104.167.102.1 dev ens33  proto static  metric 1024
10.172.1.1 via 10.172.1.5 dev tun0
10.172.1.5 dev tun0  proto kernel  scope link  src 10.172.1.6
104.167.102.0/24 dev ens33  proto kernel  scope link  src 104.167.102.77
109.201.154.177 via 104.167.102.1 dev ens33
128.0.0.0/1 via 10.172.1.5 dev tun0
odie5533
  • 445
  • 1
  • 4
  • 7

8 Answers8

36

Based on @MrK answer, I've write some simple code here to make it quicker to do the job so you doesn't have to check for interfaces/IP:

ip rule add from $(ip route get 1 | grep -Po '(?<=src )(\S+)') table 128
ip route add table 128 to $(ip route get 1 | grep -Po '(?<=src )(\S+)')/32 dev $(ip -4 route ls | grep default | grep -Po '(?<=dev )(\S+)')
ip route add table 128 default via $(ip -4 route ls | grep default | grep -Po '(?<=via )(\S+)')

I've tried this script on 4 of my VPS and it's working perfectly.

minhng99
  • 503
  • 4
  • 9
21

This may be a bit late, but ...

The problem is that the default gateway gets changed by OpenVPN, and that breaks your current SSH connection unless you set up appropriate routes before you start OpenVPN.

What follows works for me. It uses iptables and ip (iproute2). Below, it is assumed that the default gateway interface before OpenVPN is started is "eth0". The idea is to ensure that when a connection to eth0 is made, even if eth0 is not the default gateway interface anymore, response packets for the connection go back on eth0 again.

You could use the same number for the connection mark, firewall mark and routing table. I used distinct numbers to make the diffences between them more apparent.

# set "connection" mark of connection from eth0 when first packet of connection arrives
sudo iptables -t mangle -A PREROUTING -i eth0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

# set "firewall" mark for response packets in connection with our connection mark
sudo iptables -t mangle -A OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

# our routing table with eth0 as gateway interface
sudo ip route add default dev eth0 table 3412

# route packets with our firewall mark using our routing table
sudo ip rule add fwmark 4321 table 3412

===

UPDATE:

The above works fine for me on Debian Jessie. But on an older Wheezy system I have just found that I need to add "via" to the routing table entry:

# our routing table with eth0 as gateway interface
sudo ip route add default dev eth0 via 12.345.67.89 table 3412

There "12.345.67.89" must be the original non-VPN gateway.

John Doe
  • 311
  • 2
  • 3
  • 1
    THANK YOU! I have been looking for a solution for this problem for days, and your answer solved it for me. This should be the accepted answer. – Mike Turley Jun 27 '17 at 01:40
  • This solution also worked for me. Thank you very much for sharing this. – alecov Feb 17 '18 at 22:08
  • Can there be two routes in the routing table with the same destination (`ip route add default`)? I get "RTNETLINK answers: File exists". I'm running Ubuntu Xenial. "via" doesn't help. I just tried it on Arch Linux, first `ip route add default` seem to succeed, but `ip route` output doesn't change. Any subsequent runs result in "file exists" message. – x-yuri Apr 24 '18 at 17:20
  • I see, the route is added to 3412 routing table. And to get to it you've got to: `ip route show table all | grep 3412`. And without "via" non-established (if I'm not mistaken) connections stop to work (Ubuntu Xenial). At least I'm able to correct the routing table. But nevertheless, I can't access the server after I run `openvpn`. – x-yuri Apr 24 '18 at 17:59
  • Didn't work for me for some reason, as opposed to [this](https://serverfault.com/a/660106/162443), or [this](https://unix.stackexchange.com/a/242808/29867), or [this](https://serverfault.com/questions/868249/linux-ubuntu-openvpn-client-do-not-tunnel-ssh/868694#868694). Which are basically the same. – x-yuri Apr 26 '18 at 15:36
  • wow! thank you so much! never know iptables could do this – lurf jurv Oct 21 '19 at 17:12
19

I'm having a similar issue to this and have been attempting the fix described in this forum post.

The idea is that currently when you connect to your public IP address, the return packets are being routed over the VPN. You need to force these packets to be routed over your public interface.

These route commands will hopefully do the trick:

ip rule add from x.x.x.x table 128

ip route add table 128 to y.y.y.y/y dev ethX

ip route add table 128 default via z.z.z.z

Where x.x.x.x is your public IP, y.y.y.y/y should be the subnet of your public IP address, ethX should be your public Ethernet interface, and z.z.z.z should be the default gateway.

Note that this hasn't worked for me (using Debian and PrivateInternetAccess) but may help you out.

Community
  • 1
MrK
  • 306
  • 1
  • 4
  • 1
    Instead of just linking to a solution, please state or at least summarize the solution here. That way, your post can still be useful in the future if the post you linked to goes away. – Andrew Schulman Jan 16 '15 at 16:45
  • I've run these commands to no avail... How would I then list this table (I can't see the new rules I added with `route` or `ip route show`) and delete it? – Hussain Khalil Aug 30 '16 at 23:41
  • I lost my ssh on my public ip when i stood up openvpn on my home server. Running the first command listed above allowed me to gain ssh access back into the server when not on the home LAN. Using PIA + OpenVPN + ubuntu 16.04. – boredcoding Jun 02 '17 at 17:28
  • Does it really only route the SSH port to my public IP? Can you explain more about what this exactly do? I don't see anywhere you setting a specific port or protocol – Freedo Jun 19 '17 at 05:45
  • It doesn't route based on ports at all (128 is the table number, not a port). It's basically saying that if the packets arrive via your public IP address then the response should be sent on the same interface (without using the VPN). – MrK Jun 20 '17 at 00:56
0

I had the same issue on AWS instance(VPN Client) connected to On-premise OpenVPN server that I can't access it unless I am connected to the same VPN server or the on-premise network. The solution was just adding a route from local machine public IP address to the AWS instance gateway

ip route add xx.xxx.xx.xx via 172.xx.xx.1 dev eth0

and I can repeat same for any outside user that need to have access to the AWS instance as it is only needed to be shared with a handful of outside users.

Thesane
  • 101
  • 2
0

Expanding on https://serverfault.com/a/660106/577384 and https://serverfault.com/a/793700/577384. In order to get this to work properly with public-facing docker services, I had to also do ip -4 rule add to 172.0.0.0/8. To stop packets going to localhost (the docker services) from being routed over eth0. Making sure that this rule takes precident over the 'table' rule.

I also have the following rule before the rest of the iptable rules, for LAN devices in general iptables -A PREROUTING -t mangle -s 192.168.0.0/16 -j ACCEPT -m comment --comment "Skip special route for LAN".

(I pressum that I would need something similar for IPv6, but I don't have docker runing in IPv6 yet)

Blutti
  • 1
0

hmm sounds like ip subnet overlap.... without knowing more about your ip scheme, other than the public ip for your vpn ternmination as nl.privateinternetaccess.com, cannot tell for sure.

so for instance, if the remote subnet on the other side of nl.privateinternetaccess.com is 10.32.43.0/24, and your instance is in an aws vpc whose subnet is 10.32.44.0/24. but your source ssh client lives on 10.32.43.0/24 (your side of aws vpc), it will not work, as return ssh traffic will be pushed erroneously over vpn to the netherlands.

provide full ip / subnet info for more help with this.

...

ok, so... looks like your default route is in the tunnel, after you connect to nl:

0.0.0.0/1 via 10.172.1.5 dev tun0

so you can change that after you connect. lots of times vpn servers give you bogus routes. specially at sloppy corps. in this case, they are pushing a default route to you so all traffic from vps box is going to nl. change default route to 104.167.102.x or whatever ur subnet gateway is at ur vps provider.

nandoP
  • 2,001
  • 14
  • 15
  • What commands' outputs would suffice? – odie5533 Jan 16 '15 at 03:12
  • 1
    output of "ip addr" and "ip route" on ssh client, ssh server/vpn client, and nl vpn server. make sure vpn is up when getting output. – nandoP Jan 16 '15 at 03:17
  • I do want traffic by default to go through the VPN; however, I want incoming traffic on 104.167.102.77:22 to be allowed in so that I can SSH to the VPS. – odie5533 Jan 16 '15 at 13:26
  • you verify this with tcpdump, but it sounds like after you connect, and default route becomes tunnel, the new ssh traffic inbound from elsewhere on the internet is allowed in, but it is not returning, as the default route sends ssh response thru tunnel, instead of back to you. you can fix this by "ip route add [your-ip-addr]/32 via [your vps gateway]". to find your vps gateway, simply disconnect from tunnel, and look at default route – nandoP Jan 16 '15 at 16:00
0

When you bring the VPN up, your default gateway is replaced. This means that any traffic generated from or routed through your box will be forwarded to the VPN gateway.

A simple solution is to filter all traffic which you do not want to route through the VPN and do something else with it. One possibility is to pick up the traffic generated from your box with your local source address and route it through your local gateway. This allows services such as SSH to work properly.

We'll do that here. First, create a new routing table and add a default route which routes everything through your local gateway:

# <table> is any number between 2 and 252.
# Check /etc/iproute2/rt_tables for more info.
ip route add default via <gateway> table <table>
ip route add <lan-addr> dev <device> table <table>

Next, create a new packet filter rule that marks all traffic leaving your box from a given source address with some identifier.

# <mark> is any number.
iptables -tmangle -AOUTPUT -s<local-addr> -jMARK --set-mark <mark>

Finally, create a routing policy that picks all aforementioned marked traffic and routes it using the generated table above.

ip rule add fwmark <mark> table <table>

Once again, the values of <mark> and <table> are arbitrary identifiers of your own choice.

alecov
  • 552
  • 1
  • 6
  • 13
0

For me with running OpenVPN server myself in pfSense was to uncheck setting "Force all client-generated IPv4 traffic through the tunnel."

user3820047
  • 1