Windows 10 L2TP VPN connection issue

Question

I have a Windows 10 client which has been configured to connect to an L2TP VPN.

To get the connection to work, I had to add the "AssumeUDPEncapsulationContextOnSendRule" registry Key, and then totally disable Windows Firewall—the VPN connects perfectly this way.

However, I cannot get the VPN to connect with the Windows Firewall enabled and I really need to keep the Windows Firewall enabled and be able to connect to the L2TP VPN.

I have also allowed...

• Protocol 50 (ESP) in and out
• UDP 1701, 4500, 500 in and out

However the VPN still only connects if I totally disable the Windows Firewall and not with it enabled.

Question: Any help or guidance suggested to further troubleshoot this problem?

Answer 1

Trouble getting Windows to connect to an L2TP VPN

1. Firstly, if the VPN server is behind a NAT and the VPN client is behind a NAT this could cause a problem because apparently "by default Windows does not support IPSec network address translation (NAT) Traversal (NAT-T) security associations to servers that are located behind a NAT device", and this applies to Windows 10 still as well.

   The advice given by Microsoft "if you have to put a server behind a NAT device and then use an IPsec NAT-T environment, you can enable communication by changing a registry value on the VPN client computer and the VPN server."

   PowerShell (Suggested Fix)

   Note: You must run this in an admin elevated PowerShell session.

   ## -- Add registry key to support for L2TP communications via double NAT
   Set-ItemProperty -Path "HKLM:SYSTEM\CurrentControlSet\Services\PolicyAgent" -Name "AssumeUDPEncapsulationContextOnSendRule" -Type DWORD -Value 2 –Force;
   

   Important: You must restart the machine(s) you apply this to before it's effective.

   Create and configure the AssumeUDPEncapsulationContextOnSendRule registry key with a 2 value beneath HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent registry subkey and define it as a DWORD value.

   ---

   • A value of 2 configures Windows so that it can establish security associations when both the Windows Server and Windows VPN client computer are behind NAT devices.

   Source

   PowerShell (Remove Fix)

   Note: You must run this in an admin elevated PowerShell session.

   ## -- Remove registry key for L2TP communications support via double NAT
   Remove-ItemProperty -Path "HKLM:SYSTEM\CurrentControlSet\Services\PolicyAgent" -Name "AssumeUDPEncapsulationContextOnSendRule" –Force;
   

   Important: You must restart the machine(s) you apply this to before it's effective.

---

If it is not a double NAT issue then . . .

2. It seems perhaps in some configurations port 1701 is used over TCP and UDP both and not just UDP only. Adjust your rule to allow the TCP port 1701 through as well and see if that fixes the problem.

   Layer Two Tunneling Protocol (L2TP) uses TCP port 1701 and is an extension of the Point-to-Point Tunneling Protocol. L2TP is often used with IPSec to establish a Virtual Private Network (VPN).

   Source

3. Furthermore, ensure your Windows Firewall "allow" rule(s) for the applicable TCP and UDP ports (and any correlated VPN client software exe's, etc.) from the Advanced tab has all private, domain, and public profiles checked.

   

4. Upon further research it's not super clear to me if some of this applies to the client side rather than the VPN server side for L2TP but some advice suggests actually allowing UDP port 50.

5. There's another post that also talks about the ESP (value 50) <- Used by IPSec data path and others that refer to the ports used by IPSec protocols and ports.

   • Remember, I'm not certain if #4 and #5 both apply to the connecting client side but something to consider investigating further and opening up additional access for correlated protocols and/or ports.

Further Troubleshooting

To troubleshoot further, consider running Wireshark with the Windows Firewall disabled and make the successfully VPN connection and save that trace. Then with the Windows Firewall enabled, run a new trace, attempt a VPN connection, and save that trace.

Now you can look over both successful and unsuccessful L2TP VPN connection traces, filter, and see at the packet level what is really going on to determine what further you may need to allow through the Windows Firewall.

---

Supporting Resources

• How to configure an L2TP/IPsec server behind a NAT-T device in Windows and in Windows Server
• Windows 10 L2TP/IPsec Manual Setup Instructions
• What is a NAT firewall and how does it work?
• Set-ItemProperty
• Remove-ItemProperty
• L2TP Ports

• Understanding Firewall Profiles

• Layer 2 Tunneling Protocol

Answer 2

You've configured the wrong ports in your Windows firewall.

For L2TP you need UDP ports 500, 1701 and 4500.

Port 5000 is incorrect (if that was not a typo).

Answer 3

You might be missing IP Protocol ID 51, used for Authentication Header (AH) traffic.

From IBM Layer 2 Tunnel Protocol:

L2TP is actually a variation of an IP encapsulation protocol. The L2TP tunnel is created by encapsulating an L2TP frame inside a User Datagram Protocol (UDP) packet, which in turn is encapsulated inside an IP packet. The source and destination addresses of this IP packet define the endpoints of the connection. Because the outer encapsulating protocol is IP, you can apply IPSec protocols to the composite IP packet. This protects the data that flows within the L2TP tunnel. You can then apply Authentication Header (AH), Encapsulated Security Payload (ESP), and the Internet Key Exchange (IKE) protocol in a straightforward way.

Wikipedia IPSEC:

Authentication Headers (AH) provides connectionless data integrity and data origin authentication for IP datagrams and provides protection against replay attacks.

Answer 4

DNS (port 53 [and 853 over TLS])?

Assuming you know reasonably well what you are doing and have already ticked off all the listed required ports, I would guess you are getting plunked by DNS queries. This is the one that was getting me. I was using hostnames not IPs, so it kept trying to do a DNS lookup first (through a firewall that was blocking them).