Route traffic between two IPsec tunnels

Question

I run a backend on DO infrastructure, call it site Yvi, that connects to a third party site Prov via an IPsec tunnel, with this libreswan config:

conn prov-client
  ...
  right=$YVI_IP
  rightsourceip=10.31.3.1
  rightsubnet=10.31.3.0/28
  left=$PROV_IP
  leftsubnet=10.70.0.36/28

Prov has a server running on 10.70.0.37, and I'm able to interact with it from Yvi.

My problem is that I'm setting up a local dev environment (an Ubuntu box in another network), and each time I make a change I have to deploy to Yvi because only from there can I reach the API in Prov. I'd like to avoid this by connecting Local to Yvi and route that traffic to Prov to be able to reach the API in Prov from Local and ease development.

I connect Local to Yvi as a road warrior with the following conf:

conn remote-dev-client
  ...
  left=$YVI_IP
  leftsubnet=10.31.3.0/28
  right=%any
  rightaddresspool=10.31.4.1-10.31.4.254

Connection is established successfully and from Local I can reach 10.31.3.1 on Yvi. What I want is to reach 10.70.0.37 in Prov from Local. The route to the 10.70.0.36/28 network is not added automatically, so I tried setting some ip xfrm and ip route rules manually on Local:

# Outgoing
ip xfrm policy add dst 10.70.0.37 src 10.31.4.1 dir out tmpl src $LOCAL_IP dst $YVI_IP proto esp spi $SPI reqid $REQID mode tunnel priority 100000

# Incoming
ip xfrm policy add dst 10.31.4.1 src 10.70.0.37 dir fwd tmpl src $YVI_IP dst $LOCAL_IP proto esp reqid $REQID mode tunnel priority 100000
ip xfrm policy add dst 10.31.4.1 src 10.70.0.37 dir in tmpl src $YVI_IP dst $LOCAL_IP proto esp reqid $REQID mode tunnel priority 100000

ip route add table 220 src 10.31.4.1 10.70.0.37 via $LOCAL_IP dev $LOCAL_IF proto static

I now run ip xfrm monitor on Yvi and then from Local ping 10.70.0.37; I can see the packets arriving at Yvi (from the xfrm monitor in Yvi), but only the outgoing, not the response (as is seen if I ping 10.31.3.1, for example), suggesting that Yvi is receiving the traffic but not routing it to Prov? I really don't know how to interpret this.

I think I have to add routes in Yvi to route the traffic to the Prov API correctly, but adding similar rules to the ones above hasn't worked. I'd appreciate help in understanding what I'm missing, and what I'm doing wrong.

Suggestions for a different approach are also welcome, although the only way to connect to Prov, which I don't control, is through an IPsec tunnel from Yvi, which I do control.

To do that you'd need the route for your *local* on *prov* via *Yvi*, or NAT traffic on *Yvi* from *local* to *prov* as an IP already routed. Or just do an ssh tunnel and deploy via Yvi from local to prov. — Jacob Evans, Dec 03 '20 at 04:03
@JacobEvans I don't control Prov, so I wouldn't be able to set up the route to Local via Yvi. In that case, it would necessarily have to be NAT? — acib708, Dec 03 '20 at 22:14

score 0 · Accepted Answer · answered Dec 04 '20 at 01:07

I was able to solve it with iptables NAT rules. The ip xfrm policies were not necessary. This is a small explanation of what I did, for someone who, as me, is not an expert:

From Yvi I am assigning the road warriors a 10.31.4.0/24 subnet, so a route for that network is automatically installed by the keying daemon (libreswan in my case) so I added NAT rules in Yvi (/etc/ufw/before.rules, since I am using UFW, but you could achieve the same with iptables directly):

*nat
-F
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

# PRE 
-A PREROUTING -s 10.31.4.0/24 -d 10.31.3.2 -j DNAT --to-destination 10.70.0.37


# POST
-A POSTROUTING -s 10.31.4.0/24 -d 10.70.0.37 -j SNAT --to-source 10.31.3.1

COMMIT

The *nat tells iptables to apply the rules to the NAT table and the COMMIT actually saves the rules. The -F is just there for convenience, as UFW adds the rules on ufw enable but doesn't delete them on ufw disable so you end up with duplicates, hence the flush flag -F.

The PREROUTING rule applies to packets incoming from the road warrior subnet destined to 10.31.3.2, and what it does is it changes the destination address to be 10.70.0.37 instead of 10.31.3.2, effectively assigning that IP address to the Prov server, from the road warriors' perspective.

The POSTROUTING rule applies to packets incoming from the road warrior subnet about to go out to 10.70.0.37 (so, packets that just hit the prerouting rule), and changes their destination address from an address in 10.31.4.0/0 to 10.31.3.1. This is necessary because I don't control the routes, server, or anything in Prov, so if Prov gets a request from the 10.30.4.0/24 subnet, it wouldn't know how to respond. But it does know 10.31.3.1.

And that's it! Now I can reach Prov from Local via Yvi at 10.31.3.2.

If you're trying to debug iptables, I found it helpful to add duplicate LOG rules before each rule you're debugging, something like this: -A POSTROUTING -s 10.31.4.0/24 -d 10.70.0.37 -j LOG --log-prefix "SOME_IDENTIFIER" and now you should be able to grep SOME_IDENTIFIER from syslog, and it'll show up every time the rule matches. — acib708, Dec 04 '20 at 01:08

Route traffic between two IPsec tunnels

1 Answers1