I am trying to replace an old VMWare server (ESXI 5.1.0) with a new one (ESXI 6.7.0) and to do that we are trying to replicate the setup of the old one. We have another physical server that needs to be connected to the virtual servers.
So I am trying to set up a VNP connection between a Windows 2012R2 server (client) and a vyatta router through lt2p.
UPDATE: we are now trying with the latest version of Vyos router, but the result is the same.
Did this before multiple times and currently I am just replicating the settings on both side, based on the already working solution, but somehow this time it just does not want to connect.
The same server is already successfully connected to other two other VPNs with identical setup also using l2tp and vyatta routers.
On the vyatta side I can see the following error in the logs:
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [01528bbbc00696121849ab9a1c5b2a5100000001]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: received Vendor ID payload [MS NT5 ISAKMPOAKLEY 00000009]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: received Vendor ID payload [RFC 3947]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [FRAGMENTATION]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [MS-Negotiation Discovery Capable]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [Vid-Initial-Contact]
Apr 5 10:34:56 vyatta pluto[3061]: packet from XX.YYY.ZZZ.86:500: ignoring Vendor ID payload [IKE CGA version 1]
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: responding to Main Mode from unknown peer XX.YYY.ZZZ.86
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: Oakley Transform [AES_CBC (256), HMAC_SHA1, ECP_384] refused due to strict flag
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: Oakley Transform [AES_CBC (128), HMAC_SHA1, ECP_256] refused due to strict flag
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: Oakley Transform [AES_CBC (256), HMAC_SHA1, MODP_2048] refused due to strict flag
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: Oakley Transform [3DES_CBC (192), HMAC_SHA1, MODP_2048] refused due to strict flag
Apr 5 10:34:56 vyatta kernel: [262404.703564] [NAT-DST-2] IN=eth1 OUT= MAC=00:0c:29:0f:29:52:00:22:bd:f8:19:zz:08:00 SRC=XX.YYY.ZZZ.86 DST=VVV.MMM.WW.168 LEN=436 TOS=0x00 PREC=0x00 TTL=126 ID=28719 PROTO=UDP SPT=500 DPT=500 LEN=416
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: NAT-Traversal: Result using RFC 3947: no NAT detected
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:34:56 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:34:57 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:34:57 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:34:57 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:34:58 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:34:58 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:34:58 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:35:01 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:35:01 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:35:01 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:35:08 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:35:08 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:35:08 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:35:23 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:35:23 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:35:23 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:35:38 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: next payload type of ISAKMP Identification Payload has an unknown value: 77
Apr 5 10:35:38 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
Apr 5 10:35:38 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: sending encrypted notification PAYLOAD_MALFORMED to XX.YYY.ZZZ.86:500
Apr 5 10:36:06 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86 #6: max number of retransmissions (2) reached STATE_MAIN_R2
Apr 5 10:36:06 vyatta pluto[3061]: "remote-access-mac-zzz"[6] XX.YYY.ZZZ.86: deleting connection "remote-access-mac-zzz" instance with peer XX.YYY.ZZZ.86 {isakmp=#0/ipsec=#0}
In the log it says "next payload type of ISAKMP Identification Payload has an unknown value: 77", the value is different at every connection.
There is not much on the Windows side in the logs. It just counts the seconds without an end.
Log Name: Application
Source: RasClient
Date: 05/04/2019 10:34:56
Event ID: 20221
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: *******.******.local
Description:
CoId={104D3F70-1CB1-40A1-A229-7F7E5A943E64}: The user SYSTEM has started dialing a VPN connection using a all-user connection profile named ZZZZZZZ. The connection settings are:
Dial-in User = ******
VpnStrategy = L2TP
DataEncryption = Require
PrerequisiteEntry =
AutoLogon = No
UseRasCredentials = Yes
Authentication Type = CHAP/MS-CHAPv2
Ipv4DefaultGateway = No
Ipv4AddressAssignment = By Phonebook Entry
Ipv4DNSServerAssignment = By Phonebook Entry
Ipv6DefaultGateway = Yes
Ipv6AddressAssignment = By Server
Ipv6DNSServerAssignment = By Server
IpDnsFlags =
IpNBTEnabled = No
UseFlags = Private Connection
ConnectOnWinlogon = No
IPsec authentication for L2TP = Pre-shared key.
Log Name: Application
Source: RasClient
Date: 05/04/2019 10:34:56
Event ID: 20222
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: *******.******.local
Description:
CoId={104D3F70-1CB1-40A1-A229-7F7E5A943E64}: The user SYSTEM is trying to establish a link to the Remote Access Server for the connection named ZZZZZZZ using the following device:
Server address/Phone Number = VVV.MMM.WW.168
Device = WAN Miniport (L2TP)
Port = VPN0-3
MediaType = VPN.
And here is the configuration of the vyatta router:
interfaces {
ethernet eth0 {
address 192.168.1.254/24
duplex auto
hw-id 00:0c:29:0f:29:48
smp_affinity auto
speed auto
}
ethernet eth1 {
address VVV.MMM.WW.168/24
duplex auto
hw-id 00:0c:29:0f:29:52
smp_affinity auto
speed auto
}
loopback lo {
}
}
nat {
destination {
rule 2 {
description "IPSEC TUNNELING PORT 500"
destination {
port 500
}
inbound-interface eth1
log enable
protocol tcp_udp
translation {
port 500
}
}
rule 3 {
description "IPSEC TUNNELING PORT 4500"
destination {
port 4500
}
inbound-interface eth1
log enable
protocol tcp_udp
translation {
port 4500
}
}
rule 4 {
description "VPN CLIENT TUNNELING PORT 1701"
destination {
port 1701
}
inbound-interface eth1
log enable
protocol tcp_udp
translation {
port 1701
}
}
}
source {
rule 10 {
description "OUTSIDE CONNECTION"
outbound-interface eth1
source {
address 192.168.1.0/24
}
translation {
address masquerade
}
}
}
}
protocols {
rip {
network 192.168.1.0/24
}
static {
route 10.1.1.0/24 {
next-hop 192.168.1.1 {
}
}
route 192.168.2.0/24 {
next-hop 192.168.1.1 {
}
}
route 192.168.3.0/24 {
next-hop 192.168.1.1 {
}
}
}
}
service {
ssh {
disable-password-authentication
port 22
}
}
system {
config-management {
commit-revisions 20
}
console {
}
gateway-address VVV.MMM.WW.1
host-name vyatta
login {
user vyatta {
authentication {
encrypted-password ****************
public-keys vyatta@vyatta {
key ****************
type ssh-rsa
}
}
level admin
}
}
name-server 8.8.8.8
name-server 192.168.1.2
name-server 192.168.3.2
ntp {
server 0.vyatta.pool.ntp.org {
}
}
package {
auto-sync 1
repository community {
components main
distribution stable
password ****************
url http://packages.vyatta.com/vyatta
username ""
}
}
syslog {
global {
facility all {
level notice
}
facility protocols {
level debug
}
}
}
time-zone GMT
}
vpn {
ipsec {
ipsec-interfaces {
interface eth1
}
nat-networks {
allowed-network 10.1.1.0/24 {
}
allowed-network 192.168.1.0/24 {
}
allowed-network 192.168.2.0/24 {
}
allowed-network 192.168.3.0/24 {
}
}
nat-traversal enable
}
l2tp {
remote-access {
authentication {
local-users {
username XYZ {
password ****************
}
}
mode local
}
client-ip-pool {
start 192.168.1.100
stop 192.168.1.110
}
dns-servers {
server-1 192.168.1.2
}
ipsec-settings {
authentication {
mode pre-shared-secret
pre-shared-secret ****************
}
ike-lifetime 3600
}
outside-address VVV.MMM.WW.168
outside-nexthop 0.0.0.0
}
}
}
What I know and tried:
- checked the preshared secrets many times and the authentication, retyped it many times and I am 100% sure it is not the issue.
- read somewhere NAT can mess up the packets, but there is no NAT (as far as I know) and nat-traversal is enabled.
- tried to change the network adapter on the virtual router, tried all the possible options, same error happens
- This setup is the exact copy of the other two working connection. Checked the configuration many times for typos or wrong IP addresses, etc.
Please let me know if you have any idea what could cause this issue.
I would really appreciate any tips, ideas, even guesses about what I could check. :)
Thanks.
