1

Basically I'm trying to connect a pfSense to an EdgeRouter via IPsec site2site.
(public ip networks obfuscated by '1.2.')

             [pfsense] <-> [edgerouter]  
public: 1.2.156.229/30 <-> 1.2.112.249/30
tunnel: 10.5.44.100/24 <-> 10.20.30.100/24

IPsec settings on both sites:
phase1: IKEv2 PSK AES128 SHA1 DH2
phase2: ESP AES128 SHA1

EdgeRouter has Internet access via mesh-routed OLSR, so its gateway is commonly non-local and is also subject to change if the mesh network changes. This is intended this way by OLSR so its not wrong in this setup that the gateway is not on same subnet.

The tunnel/connection is up but there is no traffic passing through it, so after raising strongswan kernel loglevel and digging in charon.log on both sites, I found a problem with setting up routes on EdgeRouter:

charon.log on edgerouter: 

Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> getting a local address in traffic selector 10.20.30.0/24
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> using host 10.20.30.100
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> sending RTM_GETROUTE 207: => 52 bytes @ 0x711f80a8
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>    0: 34 00 00 00 1A 00 01 00 CF 00 00 00 6A 6B 00 00  4...........jk..
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   16: 02 00 00 00 00 00 00 00 00 00 00 00 08 00 10 00  ................
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   32: FF FF FF FF 08 00 07 00 4E 29 70 F9 08 00 01 00  ........N)p.....
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   48: C1 EE 9C E5                                      ....
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> received RTM_NEWROUTE 207: => 112 bytes @ 0x604f58
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>    0: 70 00 00 00 18 00 00 00 CF 00 00 00 6A 6B 00 00  p...........jk..
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   16: 02 20 00 00 FE 00 00 01 00 02 00 00 08 00 0F 00  . ..............
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   32: FE 00 00 00 08 00 01 00 C1 EE 9C E5 08 00 04 00  ................
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   48: 0A 00 00 00 08 00 07 00 4E 29 70 F9 08 00 05 00  ........N)p.....
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   64: 4E 29 76 75 08 00 10 00 FF FF FF FF 24 00 0C 00  N)vu........$...
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   80: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   96: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> using 1.2.118.117 as nexthop to reach 1.2.156.229/32
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> 1.2.112.249 is on interface br0
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> installing route: 10.5.44.0/24 via 1.2.118.117 src 10.20.30.100 dev br0
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> getting iface index for br0
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> sending RTM_NEWROUTE 208: => 60 bytes @ 0x711f8090
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>    0: 3C 00 00 00 18 00 05 06 D0 00 00 00 6A 6B 00 00  <...........jk..
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   16: 02 18 00 00 DC 04 00 01 00 00 00 00 08 00 01 00  ................
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   32: 0A 05 2C 00 08 00 07 00 0A 14 1E 64 08 00 05 00  ..,........d....
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   48: 4E 29 76 75 08 00 04 00 0A 00 00 00              N)vu........
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> received (2) 208: => 80 bytes @ 0x604fe8
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>    0: 50 00 00 00 02 00 00 00 D0 00 00 00 6A 6B 00 00  P...........jk..
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   16: FD FF FF FF 3C 00 00 00 18 00 05 06 D0 00 00 00  ....<...........
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   32: 6A 6B 00 00 02 18 00 00 DC 04 00 01 00 00 00 00  jk..............
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   48: 08 00 01 00 0A 05 2C 00 08 00 07 00 0A 14 1E 64  ......,........d
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1>   64: 08 00 05 00 4E 29 76 75 08 00 04 00 0A 00 00 00  ....N)vu........
Mar  4 23:27:27 12[KNL] <peer-1.2.156.229-tunnel-1|1> unable to install source route for 10.20.30.100
Mar  4 23:27:27 12[IKE] <peer-1.2.156.229-tunnel-1|1> CHILD_SA peer-1.2.156.229-tunnel-1{2} established with SPIs c042bc69_i c46929b0_o and TS 10.20.30.0/24 === 10.5.44.0/24
Mar  4 23:27:40 11[KNL] creating roam job due to route change
Mar  4 23:27:40 11[KNL] <peer-1.2.156.229-tunnel-1|1> sending RTM_GETROUTE 209: => 52 bytes @ 0x719f8888

I tried to reproduce the error to understand whats going wrong.

# # reproduce error:
# ip route add 10.5.44.0/24 via 1.2.118.117 src 10.20.30.100 dev br1
RTNETLINK answers: No such process

# # check default route and local ip address:
# ip route show | grep 0.0.0.0
0.0.0.0/1 via 1.2.118.117 dev br0  metric 2 onlink
# ip -f inet address show br0
10: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    inet 1.2.112.249/30 brd 1.2.112.251 scope global br0
# ip -f inet address show br1
11: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1462 qdisc noqueue state UP group default
    inet 10.20.30.100/24 brd 10.20.30.255 scope global br1

# # try to narrow down the problem
# ip route add 10.5.44.0/24 via 1.2.118.117 src 10.20.30.100 dev br1
RTNETLINK answers: No such process
# ip route add 10.5.44.0/24 src 10.20.30.100 dev br1
# ip route change 10.5.44.0/24 via 1.2.118.117 src 10.20.30.100 dev br1
RTNETLINK answers: No such process

Now I don't understand what rtnetlink is missing or what is wrong with gateway?

Searching for the strongswan or rtnetlink errors, does not give anything special as answer, just general explanations which i already understand. My next guess would be, I missed something while setting up this tunnel? The EdgeRouter has a bridge interface(br0) with public ip for internet access and a second bridge interface(br1) with local ip for mgmt network.

Also I checked this article describing IPsec on EdgeRouter and my configuration is nearly same, aside that I'm using bridge interfaces and IKEv2 (instead described IKEv1).

Digging deeper just got me to What CAN cause 'RTNETLINK answers : No such process' when adding a route and now I'm out of ideas what could be wrong.

Community
  • 1
Christoph Lösch
  • 111
  • 1
  • 7
  • AFAIK, using strongswan or openswan on Linux with LAN to LAN, there shouldn't be a route showing in your `netstat -nr`, pointing to the remote LAN. Which is confusing, as I'm used to have one with FreeBSD or OpenBSD, using racoon or isakmpd. – SYN Mar 05 '17 at 01:23
  • While: the reason you can't add a route is explained in the last link you're quoting: setting a gateway that isn't reachable in your network doesn't make sense. And you shouldn't instruct your kernel to forward traffic for your remote LAN through your public NIC anyway. – SYN Mar 05 '17 at 01:25
  • Now: why do you have these logs is weird... Can we have some idea of what your strongswan configuration looks like? – SYN Mar 05 '17 at 01:27
  • Oh, right, I forgot to mention, that the edgerouter is on a mesh network with OLSR so thats why the gateway is not on same network. Though this seems wrong in most configurations, in this configuration i cant change the fact the gateway will hardly ever be in same network. But it should be possible though AFAIK. I updated the post. What should i add about the strongswan configuration? I linked to a setup I actually want to use. – Christoph Lösch Mar 05 '17 at 02:08

1 Answers1

0

solved the problem.

as strongswan daemon wanted to install following route:

ip route add 10.5.44.0/24 via 1.2.118.117 src 10.20.30.100 dev br0

which didnt work, according to Gateway on a different subnet on Linux i've set the following two routes on edgerouter:

ip route add 10.5.44.101 dev br1
ip route add 10.5.44.0/24 via 10.5.44.101 dev br1

10.5.44.101 is the internal remote-side of the ipsec tunnel. interface br1 needs to be used because the tunnel works with it cause of the defined local network.

hth

Christoph Lösch
  • 111
  • 1
  • 7