3
2
This is probably going to be quite a lengthy post because I want to make sure I don't forget anything. This is likely a very trivial issue but I can't figure out what's wrong with my setup.
Scenario
I recently configured a VPN server running OpenVPN, currently running inside a VM in ESXi. I have based the installation on this guide, and everything seems to be working fine: I can connect to my VPN server (through port forwarding on the router) and login with certificate authentication.
What is not working
I cannot access machines on my local network (192.168.0.0
) when I am connected through the VPN.
At the moment, after successfully connecting via VPN to my home network, all the machines on my LAN are not accessible (I cannot even ping them).
Network information
The network configuration is very simple:
- My local network:
192.168.0.0
- My OpenVPN IP:
192.168.0.140
- My gateway:
192.168.0.1
- My VPN network:
10.8.0.0
- My OpenVPN VPN IP:
10.8.0.1
- The router has port forwarding configured on port
1194
What I have tried so far (high level)
I have followed three resources to try and fix this issue:
- The OpenVPN routing guide, specifically the section Using routing and OpenVPN not running on the default gateway
- This short article on how to set up routing with OpenVPN to connect to hosts on my LAN
- Another article on how to configure ESXi for this to work, because apparently, in its default configuration, ESXi can cause some problems getting this to work
- Enabled IP forwarding on the VPN server
- Enabled all the traffic to go through OpenVPN's firewall
What I have tried so far (detailed)
This is the OpenVPN server.conf
:
local 192.168.0.140
topology subnet
dev tun
proto udp
port 1194
ca /etc/openvpn/easy-rsa/keys/ca.crt
cert /etc/openvpn/easy-rsa/keys/vpnserver.crt
key /etc/openvpn/easy-rsa/keys/vpnserver.key
dh /etc/openvpn/easy-rsa/keys/dh1024.pem
server 10.8.0.0 255.255.255.0
# server and remote endpoints
ifconfig 10.8.0.1 10.8.0.2
# Add route to Client routing table for the OpenVPN Server
push "route 10.8.0.1 255.255.255.255"
# Add route to Client routing table for the OpenVPN Subnet
push "route 10.8.0.0 255.255.255.0"
# your local subnet
push "route 192.168.0.0 255.255.255.0"
# Set primary domain name server address to the SOHO Router
# If your router does not do DNS, you can use Google DNS 8.8.8.8
push "dhcp-option DNS 192.168.0.254"
# Override the Client default gateway by using 0.0.0.0/1 and
# 128.0.0.0/1 rather than 0.0.0.0/0. This has the benefit of
# overriding but not wiping out the original default gateway.
push "redirect-gateway def1"
client-to-client
duplicate-cn
keepalive 10 120
tls-auth /etc/openvpn/easy-rsa/keys/ta.key 0
cipher AES-128-CBC
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn-status.log 20
log /var/log/openvpn.log
verb 1
An example client.ovpn
:
client
dev tun
proto udp
remote <my_router_ip> 1194
resolv-retry infinite
nobind
persist-key
persist-tun
mute-replay-warnings
ns-cert-type server
key-direction 1
cipher AES-128-CBC
comp-lzo
verb 1
mute 20
Since my current router does not support static routes, unfortunately I cannot configure routing once and for all, but I have do so on each device I want to access while I am connected via VPN. So for example, on my Plex server (192.168.0.110
), I have done this:
route add -net 10.8.0.0/24 gw 192.168.0.140
So that my OpenVPN machine (192.168.0.140
) could connect the machines on the VPN network (10.8.0.0
) with those on the local network (192.168.0.0
).
I have also made sure that IP forwarding is enabled on the OpenVPN server, but uncommenting the following line from /etc/sysctl.conf
:
net.ipv4.ip_forward=1
Finally, I have added the following iptables
rules to allow all traffic from the VPN network to reach the local network:
# Allow traffic initiated from VPN to access LAN
iptables -I FORWARD -i tun0 -o eth0 -s 10.8.0.0/24 -m conntrack --ctstate NEW -j ACCEPT
# Allow established traffic to pass back and forth
iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
Finally, I have configured the virtual switch vSwitch
on ESXi not to drop promiscuous packets, as the article linked above states that in its default configuration it would drop these packets, and this could cause issues when using OpenVPN while trying to access machines on the local network.
Unless I have missed something, this should really be all that needs to be done. Unfortunately, after successfully connecting to the VPN, I am still not able to ping the machines on the 192.168.0.0
network.
Again, this is likely something quite trivial, but I am willing to offer some bounty for the correct answer anyway, because one of the reasons I set up the VPN in the first place was to be able to access my other machines without the need to forward more ports on the router.
Client machine networking info
This is the full output of ipconfig
of the machine I am using to connect (in order to "fake" connecting from a remote network, I am tethering my phone's 3G connection and connecting via Wi-Fi from Windows):
C:\Windows\System32>ipconfig
Windows IP Configuration
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::2c0e:13f0:840c:37b4%15
IPv4 Address. . . . . . . . . . . : 10.8.0.10
Subnet Mask . . . . . . . . . . . : 255.255.255.252
Default Gateway . . . . . . . . . :
Wireless LAN adapter Local Area Connection* 2:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Wireless LAN adapter Wi-Fi:
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::a5e8:546c:e046:a246%4
IPv4 Address. . . . . . . . . . . : 192.168.43.220
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.43.1
Ethernet adapter Ethernet:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Ethernet adapter VirtualBox Host-Only Network:
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::1d14:52cd:fd6a:2395%10
IPv4 Address. . . . . . . . . . . : 192.168.56.1
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
Tunnel adapter isatap.{75888664-BED0-4908-8984-4DBCF9E9BDDC}:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Tunnel adapter isatap.{6F952140-AFCD-46E4-89E3-02CDEF869C50}:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Tunnel adapter isatap.{1F6CE10F-8498-4A7B-B647-FAE7422FF030}:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Output of route print
:
C:\Windows\System32>route print
===========================================================================
Interface List
15...00 ff 1f 6c e1 0f ......TAP-Windows Adapter V9
7...12 56 f2 a5 d0 53 ......Microsoft Wi-Fi Direct Virtual Adapter
4...80 56 f2 a5 d0 53 ......Killer Wireless-N 1202 Network Adapter
3...80 fa 5b 00 d7 1f ......Realtek PCIe GBE Family Controller
10...08 00 27 00 68 59 ......VirtualBox Host-Only Ethernet Adapter
1...........................Software Loopback Interface 1
8...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
9...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
11...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #4
===========================================================================
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.43.1 192.168.43.220 25
10.8.0.4 255.255.255.252 On-link 10.8.0.6 276
10.8.0.6 255.255.255.255 On-link 10.8.0.6 276
10.8.0.7 255.255.255.255 On-link 10.8.0.6 276
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.43.0 255.255.255.0 On-link 192.168.43.220 281
192.168.43.220 255.255.255.255 On-link 192.168.43.220 281
192.168.43.255 255.255.255.255 On-link 192.168.43.220 281
192.168.56.0 255.255.255.0 On-link 192.168.56.1 276
192.168.56.1 255.255.255.255 On-link 192.168.56.1 276
192.168.56.255 255.255.255.255 On-link 192.168.56.1 276
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.56.1 276
224.0.0.0 240.0.0.0 On-link 10.8.0.6 276
224.0.0.0 240.0.0.0 On-link 192.168.43.220 281
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.56.1 276
255.255.255.255 255.255.255.255 On-link 10.8.0.6 276
255.255.255.255 255.255.255.255 On-link 192.168.43.220 281
===========================================================================
Persistent Routes:
None
IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
10 276 fe80::/64 On-link
15 276 fe80::/64 On-link
4 281 fe80::/64 On-link
10 276 fe80::1d14:52cd:fd6a:2395/128
On-link
15 276 fe80::2c0e:13f0:840c:37b4/128
On-link
4 281 fe80::a5e8:546c:e046:a246/128
On-link
1 306 ff00::/8 On-link
10 276 ff00::/8 On-link
15 276 ff00::/8 On-link
4 281 ff00::/8 On-link
===========================================================================
Persistent Routes:
None
What is the IP of your client machine? If its on the same network (
192.168.0.XXX
), it won't forward the traffic to the VPN). – heavyd – 2015-01-15T21:09:23.660@heavyd When I am connected to the VPN, I normally get IP
10.8.0.6
. Let me update the question with the full output ofipconfig
, it might be useful. – user1301428 – 2015-01-15T21:11:39.533The output of
route print
on the VPN client might also be useful. Also, what do you get when trying to ping a machine? And have you runsysctl -p
or rebooted the server after modifying/etc/sysctl.conf
? – user2313067 – 2015-01-16T00:11:43.440@user2313067 I have added the output of
route print
, and I confirm that I have also runsysctl -p
. I have also double checked runningcat /proc/sys/net/ipv4/ip_forward
– user1301428 – 2015-01-16T16:19:19.287Sorry, I should've read to the end. I'd suggest opening the iptables forwarding, without connection state between the VPN and your local network and logging the rest as a first step for trouble shooting. Also, can you log dropped packets on your router? – Mahdi – 2015-01-17T05:09:00.780
Is it possible that you redirected 192.168.0.0/24 into the tunnel with
push "route 192.168.0.0 255.255.255.0"
? – harrymc – 2015-01-18T13:33:29.203@harrymc This is my first OpenVPN server config so I must admit I might be mistaken in many things, but I thought that command pushed the route to the local network to the VPN clients.. In any case, even commenting it out I see the same behavior... – user1301428 – 2015-01-18T13:53:39.143
Well, my experience is with OpenVPN servers is exactly zero, but I'm interested in the solution, so bear with me. My other theory is that when you talk to other computers on the LAN, the packets do reach them, but as the source address of these packets is from the OpenVPN network, the only thing they know to do is to uselessly send them to the default gateway which is probably the router. The solution might either be to add static routes to all LAN hosts, or use "bridging" instead of "routing". I have never done it, but read this for more info.
– harrymc – 2015-01-18T15:02:38.407@harrymc That is my understanding as well. The workaround for this is to add a static route (either on the router, which I cannot do unfortunately, or on each LAN machine that I want to access when connected to the VPN). This is what
route add -net 10.8.0.0/24 gw 192.168.0.140
should do. – user1301428 – 2015-01-18T15:09:57.273It's clear that an outside connection thru the router using the VPN wouldn't be able to see the local network segment because of its IP address. I think you should either (1) Put VPN and local network on the same 192.168.43.xxx segment by carefully limiting the DHCP IP-ranges allocated by both the router and VPN server, or (2) Use OpenVPN bridging for both segments as in my above link, or (3) Use only VPN static IP addresses for all devices in the local network. – harrymc – 2015-01-18T16:01:53.393
@harrymc Correct, and I agree that these 3 methods will work. Also, putting the VPN server on the router would also solve this. The fact is, I want to solve this specific issue with routing because I want to learn what I have done wrong. I think I have read all online threads from people that had the same issue, tried all fixes but obviously there is still something that is missing and I can't for the life of me figure out what. I think I might just give up, forget about this and look for my next puzzle to solve. Or I could put the bounty to 500, maybe that would help too. – user1301428 – 2015-01-18T16:23:46.363
See this article for using iptables for doing the NAT.
– harrymc – 2015-01-18T18:53:17.267@harrymc yep, that's one of the articles I linked in the "What I have tried so far section" :P – user1301428 – 2015-01-18T20:33:41.090
It seems you have well-researched this problem. There might even not be any error in what you did, just some bug/gotcha in the software you use. I don't think that raising the bounty will make much of a difference. Waiting and later re-posting another bounty, if unsolved, might be more effective, or maybe on another StackExchange site. Or it might be practical to use another approach, maybe such as I have outlined above, just to make it work. – harrymc – 2015-01-18T22:18:58.373
@harrymc yeah, I will definitely look at bridging as an alternative. It would be a pity really, not to know what is wrong :) I appreciate your help though! – user1301428 – 2015-01-18T22:45:38.390
@user1301428, your routing table on the client does not include any entries for the LAN subnet (in fact none of the 3 routes from the server config were actually pushed), so those packets are not going through the VPN. Perhaps check the openVPN logs on the client and sever to see if you can see why those routes were not configured. – heavyd – 2015-01-19T08:49:46.993
@heavyd hah, good catch, I will look into that. But does this mean that, when connected to the VPN, my packets are just going through my mobile gateway and are not being encrypted? – user1301428 – 2015-01-19T08:59:22.980
Yes,but they're likely being dropped by your mobile provider since they're non-routable IPs. – heavyd – 2015-01-19T09:22:56.537
@heavyd what about browsing? I mean, browsing works just fine and I am getting my home public IP address, but is that traffic also not going through the VPN? – user1301428 – 2015-01-19T09:26:08.103
Nope. Check your ip: http://www.checkip.net/. If you're going through the VPN, it should be the same as a machine on your LAN. I'm betting it's not the same.
– heavyd – 2015-01-19T09:29:32.497@heavyd yes, when I am connected to the VPN I get my home router's public IP. – user1301428 – 2015-01-19T09:30:03.280
And your routing table hasn't changed since you posted it? The posted table has your default gateway as the WiFi gateway, with no other special routes. – heavyd – 2015-01-19T09:34:17.223
@heavyd this is interesting: I am currently on another network and I can access my local machines. The routing table is indeed correct now (the routes are all being pushed - with the exception of the 192.168.0.0 route which is not there for some reason, but I do have both the 10.8.0.0 routes). At this point I am wondering whether my mobile operator does not allow using non-default routes? – user1301428 – 2015-01-19T09:51:39.393
Those are local routes, you mobile operator should not even know you have them. It is interesting though that it is working through another network. Am assuming you're using the same client machine? – heavyd – 2015-01-19T10:10:27.200
@heavyd no, I am on another machine at the moment. Do you think it could be a problem with the machine? – user1301428 – 2015-01-19T10:15:50.513
@user1301428, I do think it has something to do with your machine. I was able to recreate your scenario successfully (so none of the same problems you experienced) using near identical configuration files. Did you ever check the client log files? Perhaps setting the
verb 3
instead of 1 might give more detail. – heavyd – 2015-01-19T20:53:25.453@heavyd what are the permissions on your OpenVPN folder? I just realized that my client is not writing any logs, and in the connection phase I see a warning saying that it is not able to open my user.log file. – user1301428 – 2015-01-21T21:02:12.767
@heavyd another interesting discovery: I am able to access all of the devices on the LAN (from the other machine, that is), even those on which I haven't manually added a static route. It was my understanding that this was necessary if OpenVPN wasn't running on the network gateway. Wizardry. – user1301428 – 2015-01-21T21:03:31.230
@user1301428, yes, that is interesting. In my setup I had to add those routes manually in order to make it work. Did you ever get the original machine working? – heavyd – 2015-01-21T21:40:12.433
@heavyd nope. I should try using the same machine but on a different network. My gut is telling me that this might be an issue with the 3G connection. Sometimes (and only when I am on 3G), I successfully connect to the VPN but cannot even get the correct IP. – user1301428 – 2015-01-21T22:31:30.843
@user1301428, that is strange. I actually tried the exact same setup, using my cell as a Wi-Fi hotspot for my laptop using OpenVPN. Everything worked great for me, so I'm kinda stumped. – heavyd – 2015-01-21T22:34:04.820
Let us continue this discussion in chat.
– heavyd – 2015-01-21T22:34:18.250@heavyd sorry, I thought I would get a notification when a message was added to the chat, but apparently this is not the case. Just so you know, I have resolved the issue; thank you very much for your help ;) – user1301428 – 2015-01-25T20:35:09.793
@user1301428, welp, glad you got it working :) – heavyd – 2015-01-25T20:49:37.390