I set up wireguard by now on one server (with NAT enabled) and on a client (ubuntu). When I don't route all the traffic via the tunnel everything works. As soon as I start routing all the traffic through the tunnel as described in https://www.wireguard.com/netns/#improved-rule-based-routing no traffic is possible anymore, I cannot even ping the server's internal tunnel address from the client.
The routing on the server obviously works. Also on the client the traffic seems to be routed properly. Packets from the client appear in the server's wireguard, but packets from the server are not appearing in the client's wireguard net. On the physical link between client and server udb traffic occurs both ways.
I also set up a client on an Android device which works fine. All the traffic gets routed through the wireguard tunnel. So it has to be a problem on my ubuntu client.
All the relevant information and analysis I made so far follows.
I did few investigations by tcpdump. Therefore I constantly pinged the following three addresses from the client: 8.8.8.8, 192.168.137.1 (server in the wireguard net) and aaa.bbb.cc.dd (the server in the real world).
The hosts are the following:
- Client in local network
192.168.1.103/24 - Client in wireguard network:
192.168.137.23/24 - Server in wireguard network:
192.168.138.1/24 - Server globally:
aaa.bbb.ccc.dd
TCPDump on the server at the wireguard interface:
11:25:18.141089 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 4997, length 64
11:25:18.141120 IP 192.168.137.1 > 192.168.137.23: ICMP echo reply, id 7162, seq 4997, length 64
11:25:18.654894 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 2625, length 64
11:25:18.654917 IP aaa.bbb.cc.dd > 192.168.137.23: ICMP echo reply, id 8772, seq 2625, length 64
11:25:18.654928 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 5023, length 64
11:25:18.658361 IP 8.8.8.8 > 192.168.137.23: ICMP echo reply, id 7116, seq 5023, length 64
11:25:19.163161 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 4998, length 64
11:25:19.163197 IP 192.168.137.1 > 192.168.137.23: ICMP echo reply, id 7162, seq 4998, length 64
11:25:19.677354 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 2626, length 64
11:25:19.677382 IP aaa.bbb.cc.dd > 192.168.137.23: ICMP echo reply, id 8772, seq 2626, length 64
11:25:19.677393 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 5024, length 64
11:25:19.680897 IP 8.8.8.8 > 192.168.137.23: ICMP echo reply, id 7116, seq 5024, length 64
11:25:20.185851 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 4999, length 64
Echo requests come in and are replied properly.
Now on the client's wireguard interface:
10:48:33.129645 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 2799, length 64
10:48:33.961380 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 2825, length 64
10:48:33.961393 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 427, length 64
10:48:34.153335 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 2800, length 64
10:48:34.985387 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 428, length 64
10:48:34.985434 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 2826, length 64
10:48:35.177511 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 2801, length 64
10:48:36.009515 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 2827, length 64
10:48:36.009539 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 429, length 64
10:48:36.201508 IP 192.168.137.23 > 192.168.137.1: ICMP echo request, id 7162, seq 2802, length 64
10:48:37.033320 IP 192.168.137.23 > aaa.bbb.cc.dd: ICMP echo request, id 8772, seq 430, length 64
10:48:37.033360 IP 192.168.137.23 > 8.8.8.8: ICMP echo request, id 7116, seq 2828, length 64
Only echo requests, no replies :(
Now udp traffic on the client's physical interface:
10:50:12.265631 IP 192.168.1.103.52635 > aaa.bbb.cc.dd.51820: UDP, length 148
10:50:12.294673 IP aaa.bbb.cc.dd.51820 > 192.168.1.103.52635: UDP, length 92
10:50:17.385691 IP 192.168.1.103.52635 > aaa.bbb.cc.dd.51820: UDP, length 148
10:50:17.422843 IP aaa.bbb.cc.dd.51820 > 192.168.1.103.52635: UDP, length 92
10:50:22.441620 IP 192.168.1.103.52635 > aaa.bbb.cc.dd.51820: UDP, length 148
10:50:22.472414 IP aaa.bbb.cc.dd.51820 > 192.168.1.103.52635: UDP, length 92
10:50:27.625664 IP 192.168.1.103.52635 > aaa.bbb.cc.dd.51820: UDP, length 148
10:50:27.748373 IP aaa.bbb.cc.dd.51820 > 192.168.1.103.52635: UDP, length 92
10:50:32.745569 IP 192.168.1.103.52635 > aaa.bbb.cc.dd.51820: UDP, length 148
So packets from and to the wireguard server.
The wireguard config on the client:
root@kaksi:~# wg
interface: wireguard
public key: eJIcjMR6/M73mPI8AvvzAr9hV2qFarMwXEWDiOH+Pzk=
private key: (hidden)
listening port: 52635
fwmark: 0x926
peer: 0bseDMk5UFx+j+6Zk8FDYv4OXakOIfZj7UTdMQPtsXM=
endpoint: aaa.bbb.cc.dd:51820
allowed ips: 0.0.0.0/0
latest handshake: 6 minutes, 34 seconds ago
transfer: 134.99 KiB received, 322.50 KiB sent
The wireguard config on the server:
interface: wg0
public key: 0bseDMk5UFx+j+6Zk8FDYv4OXakOIfZj7UTdMQPtsXM=
private key: (hidden)
listening port: 51820
peer: eJIcjMR6/M73mPI8AvvzAr9hV2qFarMwXEWDiOH+Pzk=
endpoint: <client's-global-ip>:52635
allowed ips: 192.168.137.0/24
latest handshake: 49 seconds ago
transfer: 781.02 KiB received, 791.92 KiB sent
Routing on the client:
root@kaksi:~# ip route show table all
default dev wireguard table 2468 scope link
default via 192.168.1.254 dev wlp58s0 proto dhcp metric 600
169.254.0.0/16 dev wlp58s0 scope link metric 1000
192.168.1.0/24 dev wlp58s0 proto kernel scope link src 192.168.1.103 metric 600
192.168.137.0/24 dev wireguard proto kernel scope link src 192.168.137.23 metric 50
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 192.168.1.0 dev wlp58s0 table local proto kernel scope link src 192.168.1.103
local 192.168.1.103 dev wlp58s0 table local proto kernel scope host src 192.168.1.103
broadcast 192.168.1.255 dev wlp58s0 table local proto kernel scope link src 192.168.1.103
broadcast 192.168.137.0 dev wireguard table local proto kernel scope link src 192.168.137.23
local 192.168.137.23 dev wireguard table local proto kernel scope host src 192.168.137.23
broadcast 192.168.137.255 dev wireguard table local proto kernel scope link src 192.168.137.23
::1 dev lo proto kernel metric 256 pref medium
fe80::/64 dev wlp58s0 proto kernel metric 600 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium
local fe80::b6ac:1f:5294:7a4b dev wlp58s0 table local proto kernel metric 0 pref medium
ff00::/8 dev wireguard table local metric 256 pref medium
ff00::/8 dev wlp58s0 table local metric 256 pref medium
Routing table rules on the client:
root@kaksi:~# ip rule show all
0: from all lookup local
32764: from all lookup main suppress_prefixlength 0
32765: not from all fwmark 0x926 lookup 2468
32766: from all lookup main
32767: from all lookup default
On both hosts there are no ip tables rules in place, except for the nat rule on the server.
The server is a debian buster with the wireguard packages of debian unstable (0.0.20190913-1).
The client is an ubuntu 18.04 with the wireguard packages of ubuntu eoan (0.0.20190913-1ubuntu1).
