Route traffic through IPSec tunnel with gateway host

Question

Considering the strongswan wiki, this appears to be a standard problem, but I can't get it working quite right.

Network layout

The local site (client and gateway) is under my control, the remote site (remote gateway and remote server) is not. The IPSec tunnel is a split tunnel such that only requests to the 10.10.0.0/16 subnet are sent through the IPSec tunnel.

Goal

I would like the client to communicate with the remote server, e.g. create a ssh or a smb connection.

What I already did

• I've established an IPSec tunnel between the gateway and the remote gateway.

• I've enabled ip forwarding on the gateway:

  sysctl net.ipv4.ip_forward=1
  

• I've created a NAT on the gateway:

  iptables -t nat -I POSTROUTING -m policy --pol ipsec --dir out -j ACCEPT
  iptables -t nat -A POSTROUTING -j MASQUERADE
  

• On the client I've routed the traffic through the gateway:

  ip route del default
  ip route add default via 192.168.144.4
                         # 192.168.144.4 is the gateway
  

What does work

• The IPSec tunnel is established and stable.
• When logged-in into the gateway, I can ping the remote gateway and the remote server successfully. I can also ping the client. And I can ping google.com.
• When logged-in into the client, I can ping google.com and I can ping the gateway. With tcpdump icmp on the gateway I see that the ping google.com from the client is going through the gateway.

What does not work, yet

I cannot ping the remote server from the client by its IP.

client$ ping -c 1 10.10.12.7
PING 10.10.12.7 (10.10.12.7): 56 data bytes

--- 10.10.12.7 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

From the tcpdump on the gateway, it looks like the ping is sent, but not forwarded through the tunnel:

gateway$ tcpdump icmp
13:19:18.122999 IP 192.168.144.7 > 10.10.12.7: ICMP echo request, id 15, seq 0, length 64
13:19:18.123038 IP gateway > 10.10.12.7: ICMP echo request, id 15, seq 0, length 64
13:19:18.127534 IP ac5.nue3.m-online.net > gateway: ICMP net 10.10.12.7 unreachable, length 36
13:19:18.127556 IP ac5.nue3.m-online.net > 192.168.144.7: ICMP net 10.10.12.7 unreachable, length 36

As ac5.nue3.m-online.net is the internet-service provider of the local site, I think the packet is not routed through the IPSec tunnel, but through the internet connection of the gateway, which, of course, can't reach the remote server. (If I make the IPSec tunnel a full tunnel rather than a split tunnel, I get the same result.)

Any help or insight would be really appreciated!

EDIT: ipsec statusall on the gateway

gateway > ipsec statusall
Status of IKE charon daemon (strongSwan 5.8.2, Linux 5.4.0-88-generic, x86_64):
  uptime: 7 minutes, since Oct 08 08:18:24 2021
  malloc: sbrk 3112960, mmap 0, used 1081456, free 2031504
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 4
  loaded plugins: charon test-vectors ldap pkcs11 tpm aesni aes rc2 sha2 sha1 md5 mgf1 rdrand random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ctr ccm gcm ntru drbg curl attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock unity counters
Listening IP addresses:
  192.168.144.4
Connections:
example-ipsec:  %any...vpn1.example.com  IKEv2, dpddelay=300s
example-ipsec:   local:  [example@example.com] uses pre-shared key authentication
example-ipsec:   remote: [example@example.com] uses pre-shared key authentication
example-ipsec:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
example-ipsec[1]: ESTABLISHED 7 minutes ago, 192.168.144.4[example@example.com]...<public-ip-of-the-remote-gateway>[example@example.com]
example-ipsec[1]: I: 9d7c74f670bbda86_i* c12b3b4a236b7018_r, pre-shared key reauthentication in 2 hours
example-ipsec[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
example-ipsec{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: cf66ad72_i af3c9348_o
example-ipsec{1}:  AES_CBC_256/HMAC_SHA2_256_128, 442 bytes_i (4 pkts, 434s ago), 485 bytes_o (6 pkts, 433s ago), rekeying in 38 minutes
example-ipsec{1}:   10.10.102.235/32 === 0.0.0.0/0

This is the output of ipsec statusall on the gateway after unsuccessfully sending the ping from client to the remote server. The ping from the gateway does not change the "bytes" in the output. The "bytes" in the output correspond to a ping I've sent from the gateway to the remote server.

EDIT: /etc/ipsec.conf on gateway:

# /etc/ipsec.conf

conn example-ipsec
  left = %defaultroute
  leftsourceip = %config
  leftid = "example@example.com"
  right = vpn1.example.com
  rightid = "example@example.com"
  rightsubnet = 0.0.0.0/0
  leftfirewall = yes
  installpolicy = yes
  keyexchange = ikev2
  type = tunnel
  auto = start
  leftauth = psk
  rightauth = psk
  dpdaction = clear
  dpddelay = 300s

Answer 1

Since the connection uses a virtual IP address (leftsourceip=%config, which results in 10.10.102.235/32 as local traffic selector), you have to NAT traffic to that address, not the host's physical one you get via MASQUERADE, in order to match the IPsec policies and tunnel the traffic (-I to insert it at the top):

iptables -t nat -I POSTROUTING  -j SNAT --to-source 10.10.102.235

If the virtual IP is not statically assigned (e.g. based on the client's identity) and might change, you could install/delete the SNAT rule dynamically in a custom updown script (configured via leftupdown) to which the virtual IP is passed in $PLUTO_MY_SOURCEIP.

As you originally said that this is to be a split-tunneling setup (which the remote traffic selector of 0.0.0.0/0 does not actually reflect), you could also add e.g. -d 10.10.0.0/16 to the SNAT rule to only process packets to that subnet, other traffic would not get natted and tunneled (you can keep the MASQUERADE rule for that traffic). This could also be enforced via IPsec policy (rightsubnet=10.10.0.0/16), which you then get in $PLUTO_PEER_CLIENT in the updown script.