FreeBSD major update broke vpnc connectivity, incoming ESP traffic doesn't appear unencrypted on interfaces

Question

I updated a FreeBSD box from 10.4 to 11.2-RELEASE-p4 recently and it seems to have broken the vpnc VPN connectivity.

Here's the vpnc.conf:

IPSec gateway 10.1.0.1
IPSec ID vpnuser
IPSec secret su0hoh8liNgeiT8
Xauth username vpnuser
Xauth password miuthei3Niew2ee
Nat Traversal Mode none
Noninteractive

Following the interface configuration; em0 is the hardware interface with private IP address, tun0 the vpnc's interface with a public address.

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
  inet 10.2.2.1 netmask 0xfffffe00 broadcast 10.2.3.255
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (100baseTX <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21
  groups: lo
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1412
  options=80000<LINKSTATE>
  inet x.11.11.60 --> x.11.11.60 netmask 0xffffffff
  nd6 options=29
  groups: tun
  Opened by PID 90343

So far I've found out:

• vpnc can apparently get the VPN connection up, running vpnc with --no-detach doesn't show any critical errors. I am using the same config as before with the previous FreeBSD version where it worked flawlessly. I've also tried a few versions of vpnc-scripts. I've also tested it with pf firewall rules flushed with pfctl -F all.

• pings sent from the local machine (ping 8.8.8.8) do show in tcpdump -ni tun0 as outgoing traffic:

  00:58:24.017976 IP x.11.11.60 > 8.8.8.8: ICMP echo request, id 42593, seq 4, length 64
  

• pings sent from the local machine show in tcpdump -ni em0; interesting is that the VPN packet seems to get a proper response each time and the response reaches the hardware interface of local machine:

  00:58:24.018029 IP 10.2.2.1 > 10.1.0.1: ESP(spi=0x1bcc60be,seq=0x3c), length 132
  00:58:24.078558 IP 10.1.0.1 > 10.2.2.1: ESP(spi=0xe48f7620,seq=0x6b), length 132
  

• however, the returning packet doesn't show up in tcpdump.

• ping packets from a random external (internet) host to x.11.11.60 do induce similar traffic which can be seen on em0 but not on tun0:

  01:35:32.612015 IP 10.1.0.1 > 10.2.2.1: ESP(spi=0xe48f7620,seq=0x124), length 132
  

Changing sysctl value of net.inet.ip.forwarding doesn't seem to have any effect.

VPN (tun0) should be the host's default route out. Based on the finding that the example ping gets a response back til em0 this doesn't seem to be a routing problem.

Can you spot something I'm missing? Any ideas how could I get the VPN connection working again?

---

UPDATE – New findings:

It now seems probable that this is not a vpnc specific problem. Rather there might be something with ESP handling on FreeBSD 11.

• I found a workaround to the problem which is simply to force the NAT traversal mode with --natt-mode force-natt despite there's no NAT between the hosts. For some reason there's no problem with encapsulated UDP:

What shows on em0...

14:15:18.500251 IP 10.2.2.1.4500 > 10.1.0.1.4500: UDP-encap: ESP(spi=0x66842bb7,seq=0x3), length 132
14:15:18.527137 IP 10.1.0.1.4500 > 10.2.2.1.4500: UDP-encap: ESP(spi=0x3a4661f0,seq=0x3), length 132

... can be seen unencrypted now also on tun0:

14:15:18.500200 IP x.11.11.60 > 172.217.21.142: ICMP echo request, id 64016, seq 2, length 64
14:15:18.527188 IP 172.217.21.142 > x.11.11.60: ICMP echo reply, id 64016, seq 2, length 64

• I set up a separate solution with racoon using FreeBSD manual and it showed similar behaviour when it comes to apparently not handling incoming ESP packets. For some reason I get now the error vpnc[3372]: esp sendto: Invalid argument when I try to ping if vpnc was started with --natt-mode none.

• It seems there has been some changes to IPsec, ESP, and NAT-T in FreeBSD 11.0R and 11.1R. Maybe those changes interfere with something now.

Any help is still appreciated.