AWS EC2 - OpenVPN - VPN OK But No Internet Connection for Clients

Question

I've been on this for days and have tried everything I can search on the web, but nothing still seemed to work.

Clients can connect to and receive ping responses from the VPN server, and I don't see any errors in the logs. It's just that clients don't have internet connection.

Whenever I comment out push "redirect-gateway def1 bypass-dhcp" on server.conf things go fine but internet is not filtered. If it is enabled, sites would connect but would not be able to retrieve content.

On client: $ wget google.com

--2016-10-30 13:39:44--  http://google.com/
Resolving google.com (google.com)... 216.58.217.174
Connecting to google.com (google.com)|216.58.217.174|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.google.com/ [following]
--2016-10-30 13:39:44--  http://www.google.com/
Resolving www.google.com (www.google.com)... 216.58.217.164
Connecting to www.google.com (www.google.com)|216.58.217.164|:80... connected.
HTTP request sent, awaiting response...

... and stays there until timeout.

Here are my confs and settings:

System:

Ubuntu 16.04.1 on EC2, OpenVPN 2.3.10 x86_64

AWS EC2 Settings:

Security Group: Allow Inbound: ports 22/tcp and 1194/udp
                Allow Outbound: all

Source/Destination Check: Disabled # (Is this really necessary?)

Server:

$ cat /etc/openvpn/server.conf | grep -v '#' | grep -v ';' | tr -s '\n'

port 1194
proto udp
dev tun
ca ca.crt
cert AWS-01.crt
dh dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "route 10.0.0.0 255.255.0.0"
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"
client-to-client
keepalive 10 120
key-direction 0
auth SHA256
comp-lzo
max-clients 10
user nobody
group nogroup
persist-key
persist-tun
status openvpn-status.log
verb 3

$ cat /proc/sys/net/ipv4/ip_forward

$ sudo cat /etc/ufw/before.rules

...

# First entry
*nat
:POSTROUTING ACCEPT [0:0] 
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
COMMIT

...

$ cat /etc/default/ufw | grep -v '#' | tr -s '\n'

IPV6=yes
DEFAULT_INPUT_POLICY="DROP"
DEFAULT_OUTPUT_POLICY="ACCEPT"
DEFAULT_FORWARD_POLICY="ACCEPT"
DEFAULT_APPLICATION_POLICY="SKIP"
MANAGE_BUILTINS=no
IPT_SYSCTL=/etc/ufw/sysctl.conf
IPT_MODULES="nf_conntrack_ftp nf_nat_ftp nf_conntrack_netbios_ns"

$ sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
22                         LIMIT       Anywhere                  
1194/udp                   LIMIT       Anywhere                  
22 (v6)                    LIMIT       Anywhere (v6)             
1194/udp (v6)              LIMIT       Anywhere (v6)

Link: Pastebin of server's $ sudo iptables -L -v -n

Clients:

client.ovpn tested on both Ubuntu 16.10 and Android 7.0 with the same results.

client
dev tun
proto udp
remote XXX.XXX.public.ip 1194
resolv-retry infinite
nobind
user nobody
group nogroup
persist-key
persist-tun
remote-cert-tls server
cipher AES-128-CBC
auth SHA256
key-direction 1
comp-lzo
verb 3
script-security 2
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

<ca>
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
</ca>
<cert>
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN ENCRYPTED PRIVATE KEY-----
-----END ENCRYPTED PRIVATE KEY-----
</key>
<tls-auth>
-----BEGIN OpenVPN Static key V1-----
-----END OpenVPN Static key V1-----
</tls-auth>

Any help on what I'm missing? Thanks!

Additional information:

Client, pre and post VPN firewall rules:
http://pastebin.com/vJRRwzpe, http://pastebin.com/DJ6Wv5q0

Client routing table $ route -n:
preVPN:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.254.254 0.0.0.0         UG    600    0        0 wlp3s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 docker0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.254.0   0.0.0.0         255.255.255.0   U     600    0        0 wlp3s0

postVPN:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.8.0.5        128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.254.254 0.0.0.0         UG    600    0        0 wlp3s0
10.0.0.0        10.8.0.5        255.255.0.0     UG    0      0        0 tun0
10.8.0.0        10.8.0.5        255.255.255.0   UG    0      0        0 tun0
10.8.0.5        0.0.0.0         255.255.255.255 UH    0      0        0 tun0
52.45.164.133   192.168.254.254 255.255.255.255 UGH   0      0        0 wlp3s0
128.0.0.0       10.8.0.5        128.0.0.0       UG    0      0        0 tun0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 docker0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.254.0   0.0.0.0         255.255.255.0   U     600    0        0 wlp3s0

Client preVPN: http://pastebin.com/vJRRwzpe postVPN: http://pastebin.com/DJ6Wv5q0 I tried to do a `meld` comparison on both outputs and it seems that the only difference is on the `pkts` and `bytes` number... — Majal, Oct 30 '16 at 08:03
And I'm not sure if this would be interesting too (diff bet http and https): $ wget https://www.google.com --2016-10-30 15:57:56-- https://www.google.com/ Resolving www.google.com (www.google.com)... 216.58.217.164 Connecting to www.google.com (www.google.com)|216.58.217.164|:443... connected. ^C $ wget http://www.google.com --2016-10-30 15:58:19-- http://www.google.com/ Resolving www.google.com (www.google.com)... 216.58.217.164 Connecting to www.google.com (www.google.com)|216.58.217.164|:80... connected. HTTP request sent, awaiting response... ^C — Majal, Oct 30 '16 at 08:05
These are firewall rules, not routing tables. Please add all additional information to the question itself, it is not easy to read in comments. — Tero Kilkanen, Oct 30 '16 at 13:20
Sorry for that. Thanks for taking a look on this. I've done that now... — Majal, Oct 30 '16 at 13:42
Not a solution, but a troubleshooting step: add `mssfix 1400` to the openvpn configuration, and see if you get different behavior. If not, try `fragment 1400` on both client and server. I've never needed this with OpenVPN but it sounds very much like you have an MTU issue. — Michael - sqlbot, Oct 30 '16 at 13:43
Guys, thanks so much for all your help. I've been learning along the way with proper nudges from the information you are giving. This is my first install of VPN. Now it's working. The solution? Change from UDP to TCP. Yes, there could have been MTU issues, but TCP didn't really seem to care at all, taking note of MTU tables. Thanks for all your great help and time! — Majal, Oct 30 '16 at 14:55
Well, that's an option. Unfortunately, TCP comes with its own set of misery, which can vary from "I didn't really notice" to "this is so unbearably slow and the jitter is through the roof!" If it works for you, go with it, but your interests might be best served by solving the UDP issue, now, rather than later, when you decide against TCP. Note also that the potential TCP issues are not defects in OpenVPN but limitations in TCP itself and some of the ways it achieves reliability. I have some tunnels I use with TCP every day, no problem, and others that are painful without UDP. — Michael - sqlbot, Oct 30 '16 at 15:32
Sounds good. I'll sure learn lots more along the way. I'll experiment more when I'll have the time. Do you have suggested resources I can start from? Thanks for your time by the way. — Majal, Oct 30 '16 at 15:47

score 1 · Answer 1 · answered Mar 16 '20 at 20:14

On Ubuntu 18.04 you have to:

install openvpn-systemd-resolved package:

   sudo apt install openvpn-systemd-resolved

add into your .ovpn configuration file:

    script-security 2
    up /etc/openvpn/update-systemd-resolved
    down /etc/openvpn/update-systemd-resolved
    down-pre
    dhcp-option DOMAIN-ROUTE .

check in AWS console a Client VPN endpoint configuration, do you have filled DNS server IP

score 1 · Answer 2 · answered Nov 03 '21 at 09:26

1

You should enable Split Tunnel on your VPN Endpoint.

answered Nov 03 '21 at 09:26

x23b5

111
2

score 1 · Answer 3 · answered Oct 30 '16 at 07:40

1

Since you can fetch the page at google.com, which returns the redirect to www.google.com, you have nothing wrong in your configuration. This fact means that the client is able to open a TCP connection to google.com, and issue an HTTP request and receive a response.

Does this happen on other sites too?

answered Oct 30 '16 at 07:40

Tero Kilkanen

34,499
3
38
58

Hi. Yup happens in all sites I tested. It gets stuck on that last part and the page never shows up... – Majal Oct 30 '16 at 07:43
2

@TeroKilkanen good catch on the redirect. That first response would be relatively small, the next one likely larger... which suggests an MTU issue. – Michael - sqlbot Oct 30 '16 at 13:31
@Michael-sqlbot And I think there's MTU issue. @Kilkanen But what the `128.0.0.0` destination mask in default route to VPN when VPN is on? I think it should be `0.0.0.0`. – Mikhail Khirgiy Oct 30 '16 at 15:14
@MikhailKhirgiy it looks suspicious at first, but 0.0.0.0/1 and 128.0.0.0/1 are alternate default routes that simplify the process of sending all traffic over the VPN. These are expected with [`redirect-gateway def1`](http://serverfault.com/a/312977/153161). – Michael - sqlbot Oct 30 '16 at 15:25

AWS EC2 - OpenVPN - VPN OK But No Internet Connection for Clients

3 Answers3