15

I have just set up a vpn tunnel site-to-site with strongswan (4.5). The tunnel looks fine and connected to the other side, but seems there is a problem routing traffic through the tunnel.

Any idea?

Thanks!

Network diagram

+----------------------------------+
|Dedicated server: starfleet       |                   +-----------------+
|                                  |                   |  CISCO ASA      |
|        +-------------------------|     internet      |                 |
|        |eth0: XX.XX.XX.195/29    +-------------------|  YY.YYY.YYY.155 |
|        +-------------------------|                   +------+----------+
|        |virbr1: 192.168.100.1/24 |                          |
|        +----+--------------------|                          |
|             |                    |                          |
|             |                    |                   +-----------------+
|             |                    |                   |network          |
|     +-------+                    |                   |                 |
|     |                            |                   |172.30.20.0/27   |
|     |                            |                   +-----------------+
| +------------------------------+ |
| | kvm server: enterprise       | |
| |                              | |
| |                              | |
| | eth0: 192.168.100.100/24     | |
| +------------------------------+ |
+----------------------------------+

Software

  • debian wheezy
  • strongswan 4.5.2-1.5+deb7u1
  • kvm and libvirt (192.168.100.x network)

/etc/ipsec.conf

root@starfleet ~ # cat /etc/ipsec.conf
# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
    plutodebug="all"
    plutostderrlog=/var/log/pluto-ipsec
    charonstart=no
    plutostart=yes

conn net-net
     ikelifetime=86400s
     keylife=3600s
     rekeymargin=3m
     keyingtries=1
     keyexchange=ikev1
     authby=secret
     ike=aes256-sha-modp1024!
     esp=aes256-sha
     right=YY.YYY.YYY.155
     rightsubnet=172.30.20.0/27
     left=XX.XX.XX.195
     leftsubnet=192.168.100.0/24
     leftfirewall=yes
     pfs=no
     auto=add

Ipsec up

root@starfleet ~ # ipsec up net-net
002 "net-net" #1: initiating Main Mode
102 "net-net" #1: STATE_MAIN_I1: initiate
003 "net-net" #1: ignoring Vendor ID payload [FRAGMENTATION c0000000]
104 "net-net" #1: STATE_MAIN_I2: sent MI2, expecting MR2
003 "net-net" #1: ignoring Vendor ID payload [Cisco-Unity]
003 "net-net" #1: received Vendor ID payload [XAUTH]
003 "net-net" #1: ignoring Vendor ID payload [###############################]
003 "net-net" #1: ignoring Vendor ID payload [Cisco VPN 3000 Series]
106 "net-net" #1: STATE_MAIN_I3: sent MI3, expecting MR3
002 "net-net" #1: Peer ID is ID_IPV4_ADDR: 'YY.YYY.YYY.155'
002 "net-net" #1: ISAKMP SA established
004 "net-net" #1: STATE_MAIN_I4: ISAKMP SA established
002 "net-net" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+UP {using isakmp#1}
110 "net-net" #2: STATE_QUICK_I1: initiate
002 "net-net" #2: sent QI2, IPsec SA established {ESP=>0x8a12ab22 <0xa01abba1}
004 "net-net" #2: STATE_QUICK_I2: sent QI2, IPsec SA established {ESP=>0x8a12ab22     <0xa01abba1}

root@starfleet ~ # ipsec status
000 "net-net":     192.168.100.0/24===XX.XX.XX.195[XX.XX.XX.195]...YY.YYY.YYY.155[YY.YYY.YYY.155]===172.30.20. 0/27; erouted; eroute owner: #2
000 "net-net":   newest ISAKMP SA: #1; newest IPsec SA: #2; 
000 
000 #2: "net-net" STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 3331s; newest IPSEC; eroute owner
000 #2: "net-net" esp.8a12ab22@YY.YYY.YYY.155 (0 bytes) esp.a01abba1@XX.XX.XX.195 (0 bytes); tunnel
000 #1: "net-net" STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 86050s; newest ISAKMP
000

Network information

tun0 interface is used by openvpn server.

virbr1 interface is a kvm network

root@starfleet ~ # ip -4 a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet XX.XX.XX.195/29 brd XX.XX.XX.199 scope global eth0
3: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 100
    inet 10.8.0.1 peer 10.8.0.2/32 scope global tun0
5: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    inet 192.168.100.1/24 brd 192.168.100.255 scope global virbr1

root@starfleet ~ # ip -4 r s t 0
default via XX.XX.XX.193 dev eth0 
10.8.0.0/16 via 10.8.0.2 dev tun0 
10.8.0.2 dev tun0  proto kernel  scope link  src 10.8.0.1 
XX.XX.XX.192/29 via XX.XX.XX.193 dev eth0 
XX.XX.XX.192/29 dev eth0  proto kernel  scope link  src XX.XX.XX.195 
192.168.100.0/24 dev virbr1  proto kernel  scope link  src 192.168.100.1 
local 10.8.0.1 dev tun0  table local  proto kernel  scope host  src 10.8.0.1 
broadcast XX.XX.XX.192 dev eth0  table local  proto kernel  scope link  src XX.XX.XX.195 
local XX.XX.XX.195 dev eth0  table local  proto kernel  scope host  src XX.XX.XX.195 
broadcast XX.XX.XX.199 dev eth0  table local  proto kernel  scope link  src XX.XX.XX.195 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.100.0 dev virbr1  table local  proto kernel  scope link  src 192.168.100.1 
local 192.168.100.1 dev virbr1  table local  proto kernel  scope host  src 192.168.100.1 
broadcast 192.168.100.255 dev virbr1  table local  proto kernel  scope link  srr 192.168.100.1

root@starfleet ~ # ip xfrm state
src XX.XX.XX.195 dst YY.YYY.YYY.155
    proto esp spi 0x8a12ab22 reqid 16384 mode tunnel
    replay-window 32 flag af-unspec
    auth-trunc hmac(sha1) 0x######################################## 96
    enc cbc(aes) 0x################################################################
src YY.YYY.YYY.155 dst XX.XX.XX.195
    proto esp spi 0xa01abba1 reqid 16384 mode tunnel
    replay-window 32 flag af-unspec
    auth-trunc hmac(sha1) 0x######################################## 96
    enc cbc(aes) 0x################################################################

root@starfleet ~ # ip xfrm policy
src 192.168.100.0/24 dst 172.30.20.0/27 
    dir out priority 1847 ptype main 
    tmpl src XX.XX.XX.195 dst YY.YYY.YYY.155
        proto esp reqid 16384 mode tunnel
src 172.30.20.0/27 dst 192.168.100.0/24 
    dir fwd priority 1847 ptype main 
    tmpl src YY.YYY.YYY.155 dst XX.XX.XX.195
        proto esp reqid 16384 mode tunnel
src 172.30.20.0/27 dst 192.168.100.0/24 
    dir in priority 1847 ptype main 
    tmpl src YY.YYY.YYY.155 dst XX.XX.XX.195
        proto esp reqid 16384 mode tunnel
src ::/0 dst ::/0 
    socket out priority 0 ptype main 
src ::/0 dst ::/0 
    socket in priority 0 ptype main 
src ::/0 dst ::/0 
    socket out priority 0 ptype main 
src ::/0 dst ::/0 
    socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
    socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
    socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
    socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
    socket in priority 0 ptype main 

root@starfleet ~ # ip route show table 220
root@starfleet ~ # 

root@starfleet ~ # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         XX.XX.XX.193    0.0.0.0         UG    0      0        0 eth0
10.8.0.0        10.8.0.2        255.255.0.0     UG    0      0        0 tun0
10.8.0.2        0.0.0.0         255.255.255.255 UH    0      0        0 tun0
XX.XX.XX.192    XX.XX.XX.193    255.255.255.248 UG    0      0        0 eth0
XX.XX.XX.192    0.0.0.0         255.255.255.248 U     0      0        0 eth0
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr1

Iptables

root@starfleet ~ # iptables-save 
# Generated by iptables-save v1.4.14 on Fri May 24 16:07:39 2013
*nat
:PREROUTING ACCEPT [11:368]
:INPUT ACCEPT [1:48]
:OUTPUT ACCEPT [13:1012]
:POSTROUTING ACCEPT [13:1012]
-A POSTROUTING -s 10.8.0.0/16 ! -d 10.8.0.0/16 -o virbr1 -j MASQUERADE
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE
COMMIT
# Completed on Fri May 24 16:07:39 2013
# Generated by iptables-save v1.4.14 on Fri May 24 16:07:39 2013
*mangle
:PREROUTING ACCEPT [271:19504]
:INPUT ACCEPT [261:19184]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [181:28686]
:POSTROUTING ACCEPT [181:28686]
-A POSTROUTING -o virbr1 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri May 24 16:07:39 2013
# Generated by iptables-save v1.4.14 on Fri May 24 16:07:39 2013
*filter
:INPUT ACCEPT [46:3380]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [36:5220]
-A INPUT -i virbr1 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr1 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr1 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr1 -p tcp -m tcp --dport 67 -j ACCEPT
-A FORWARD -s 172.30.20.0/27 -d 192.168.100.0/24 -i eth0 -m policy --dir in --pol ipsec --reqid 16384 --proto esp -j ACCEPT
-A FORWARD -s 192.168.100.0/24 -d 172.30.20.0/27 -o eth0 -m policy --dir out --pol ipsec --reqid 16384 --proto esp -j ACCEPT
-A FORWARD -s 10.8.0.0/16 -o virbr1 -j ACCEPT
-A FORWARD -i virbr1 -o tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -d 192.168.100.0/24 -o virbr1 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.100.0/24 -i virbr1 -j ACCEPT
-A FORWARD -i virbr1 -o virbr1 -j ACCEPT
-A FORWARD -o virbr1 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr1 -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Fri May 24 16:07:39 2013

TcpDumping from 192.168.100.100 to 172.30.20.9

All commands are running at the same time.

root@enterprise:~# ping 172.30.20.9
PING 172.30.20.9 (172.30.20.9) 56(84) bytes of data.
^C
--- 172.30.20.9 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 4999ms

root@enterprise:~# tcpdump -v -n dst net 172.30.20.0/27
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:23:48.919819 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 1, length 64
16:23:49.918949 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 2, length 64
16:23:50.918950 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 3, length 64
16:23:51.918952 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 4, length 64
16:23:52.918954 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 5, length 64
16:23:53.918951 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.100.100 > 172.30.20.9: ICMP echo request, id 2605, seq 6, length 64

root@starfleet ~ # tcpdump -v -n dst net 172.30.20.0/27
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:23:50.475100 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 1, length 64
16:23:51.474262 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 2, length 64
16:23:52.474280 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 3, length 64
16:23:53.474251 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 4, length 64
16:23:54.474213 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 5, length 64
16:23:55.474173 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
XX.XX.XX.195 > 172.30.20.9: ICMP echo request, id 2605, seq 6, length 64
telemaco
  • 251
  • 1
  • 2
  • 4

2 Answers2

12

The output of the tcpdump session on starfleet reveals the problem. Due to the NAT rule here

-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE

the ICMP request with source address 192.168.100.100 gets natted to xx.xx.xx.195. As the negotiated IPsec policy is for traffic from 192.168.100.0/24 and not xx.xx.xx.195 these packets won't get encrypted. As can be seen in this schematic of the packet flow through Netfilter the rules in the POSTROUTING chain in the nat table are applied before any lookups for IPsec transforms (xfrm lookup).

To fix this do one of the following:

  • Explicitly exclude traffic to the target subnet from the MASQUERADE rules (! -d 172.30.20.0/27)

  • Add an explicit exemption rule before the MASQUERADE rules

    -A POSTROUTING -s 192.168.100.0/24 -m policy --dir out --pol ipsec -j ACCEPT

  • Leave the MASQUERADE rules as they are but configure leftsubnet=xx.xx.xx.195/32 instead (requires adjusting the configuration on the Cisco ASA box, and does not help if a site-to-site tunnel is actually your goal)

ecdsa
  • 3,800
  • 12
  • 26
  • Thanks for your response. I've tried the three advises, one by one, and it doesn't works. I've downed all interfaces except eth0 (i've cleaned iptables too) and i've applied the 3rd advice and the traffic is no routed. Any idea? – telemaco May 27 '13 at 08:50
  • Finally configuring leftsubnet=xx.xx.xx.195/32 and some nat transversal configuration in the cisco asa side, IT WORKS! :) Thanks – telemaco May 27 '13 at 09:30
  • After 8(!) Years: this answer was still relevant to me! I was just missing a single POSTROUTING rule. Thank you! – Daywalker May 02 '21 at 15:50
2

My situation is very similar to the one described by @telemaco. I have some test VMs running on KVM on my laptop computer. My laptop receives its IP address via DHCP, thus the VPN endpoint IP address is assigned by Strongswan to my laptop via leftsourceip=%config.

The VMs use a private network 192.168.100.0/24. My laptop (KVM host) receives the IP address 192.168.50.2/24 via DHCP and an endpoint IP address 10.0.0.10/26 from Strongswan.

The VMs should access a network 192.168.0.0/24, which is routed through the VPN.

Based on the answer provided by @ecdsa I got this working by adding the following rule:

-t nat -I POSTROUTING -s 192.168.100.0/24 -d 192.168.0.0/24 -j SNAT --to-source 10.0.0.10

In my case ip xfrm policy looks like this (extract):

src 192.168.0.0/24 dst 10.0.0.10/32 
    dir fwd priority 2851 
    tmpl src xx.xx.xx.xx dst 192.168.50.2
        proto esp reqid 5 mode tunnel
src 192.168.0.0/24 dst 10.0.0.10/32 
    dir in priority 2851 
    tmpl src xx.xx.xx.xx dst 192.168.50.2
        proto esp reqid 5 mode tunnel
src 10.0.0.10/32 dst 192.168.0.0/24 
    dir out priority 2851 
    tmpl src 192.168.50.2 dst xx.xx.xx.xx
        proto esp reqid 5 mode tunnel

That means only the local IP address 10.0.0.10 has a corresponding xfrm lookup rule. That's why the NAT is required, unless the VM subnet is added to IPsec.

Matthias Herlitzius
  • 121
  • 1
  • 3