3

I'm trying to set up a docker based application on a virtual server running Ubuntu. The application is supposed to work with IPv6. No problem, I got a /64 network from my server provider, it is working fine until I start the IPv6-enabled docker network.

Once the network is running, my server can still be accessed via IPv6 from the outside, but I can't connect to the internet via IPv6 from the inside - obviously there's a routing issue.

I found that the problem is the default docker0 bridge with its address fe80::1. Unfortunately the default gateway (assigned by my server provider) is also at the address fe80::1, so once the bridge is online, nothing is routed to the internet anymore.

I've been trying the entire afternoon to make docker use a different IPv6 address for the docker0 bridge (which I figure should solve my problem), but with no luck. There is an option --bip in dockerd, but unfortunately it only works with IPv4, and there is no --bip-v6 option. I also tried the --fixed-cidr-v6 option with another subnet, but that only added an additional address to the bridge without removing fe80::1.

Am I missing something? Is there a way to use another IPv6 address for the bridge interface? Or maybe a completely different solution to my problem?

This is my IPv6 routing table:
ip -6 r s table all:

local ::1 dev lo proto kernel metric 256 pref medium
2a01:4f8:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
fd4d:6169:6c63:6f77::/64 dev br-cc7dcdf95b47 proto kernel metric 256 pref medium
fd4d:6169:6c63:6f77::/64 dev br-cc7dcdf95b47 metric 1024 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev docker0 proto kernel metric 256 linkdown pref medium
fe80::/64 dev br-cc7dcdf95b47 proto kernel metric 256 pref medium
fe80::/64 dev veth048588a proto kernel metric 256 pref medium
fe80::/64 dev veth33a8cad proto kernel metric 256 pref medium
fe80::/64 dev vethe09ac37 proto kernel metric 256 pref medium
fe80::/64 dev veth1bcf186 proto kernel metric 256 pref medium
fe80::/64 dev veth2c1c3f6 proto kernel metric 256 pref medium
fe80::/64 dev veth5ae2bda proto kernel metric 256 pref medium
fe80::/64 dev veth67e374a proto kernel metric 256 pref medium
fe80::/64 dev vethb29c88d proto kernel metric 256 pref medium
fe80::/64 dev veth0d84748 proto kernel metric 256 pref medium
fe80::/64 dev vethdb1c15b proto kernel metric 256 pref medium
fe80::/64 dev vethe114d26 proto kernel metric 256 pref medium
fe80::/64 dev veth0bf244b proto kernel metric 256 pref medium
fe80::/64 dev vethdd92ee9 proto kernel metric 256 pref medium
fe80::/64 dev vethd5f5a74 proto kernel metric 256 pref medium
fe81::/64 dev docker0 proto kernel metric 256 linkdown pref medium
fe81::/64 dev docker0 metric 1024 linkdown pref medium
default via fe80::1 dev eth0 metric 1024 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium
anycast 2a01:4f8:xxxx:: dev eth0 table local proto kernel metric 0 pref medium
local 2a01:4f8:xxxx::1 dev eth0 table local proto kernel metric 0 pref medium
anycast fd4d:6169:6c63:6f77:: dev br-cc7dcdf95b47 table local proto kernel metric 0 pref medium
local fd4d:6169:6c63:6f77::1 dev br-cc7dcdf95b47 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0 table local proto kernel metric 0 pref medium
anycast fe80:: dev br-cc7dcdf95b47 table local proto kernel metric 0 pref medium
anycast fe80:: dev veth048588a table local proto kernel metric 0 pref medium
anycast fe80:: dev veth33a8cad table local proto kernel metric 0 pref medium
anycast fe80:: dev vethe09ac37 table local proto kernel metric 0 pref medium
anycast fe80:: dev veth1bcf186 table local proto kernel metric 0 pref medium
anycast fe80:: dev veth2c1c3f6 table local proto kernel metric 0 pref medium
anycast fe80:: dev veth5ae2bda table local proto kernel metric 0 pref medium
anycast fe80:: dev veth67e374a table local proto kernel metric 0 pref medium
anycast fe80:: dev vethb29c88d table local proto kernel metric 0 pref medium
anycast fe80:: dev veth0d84748 table local proto kernel metric 0 pref medium
anycast fe80:: dev vethdb1c15b table local proto kernel metric 0 pref medium
anycast fe80:: dev vethe114d26 table local proto kernel metric 0 pref medium
anycast fe80:: dev veth0bf244b table local proto kernel metric 0 pref medium
anycast fe80:: dev vethdd92ee9 table local proto kernel metric 0 pref medium
anycast fe80:: dev vethd5f5a74 table local proto kernel metric 0 pref medium
local fe80::1 dev br-cc7dcdf95b47 table local proto kernel metric 0 pref medium
local fe80::42:47ff:fe7f:2c49 dev br-cc7dcdf95b47 table local proto kernel metric 0 pref medium
local fe80::43:2cff:fe5c:bb6b dev vethdd92ee9 table local proto kernel metric 0 pref medium
local fe80::fa:2aff:fe49:e066 dev vethb29c88d table local proto kernel metric 0 pref medium
local fe80::140f:77ff:fe9b:888 dev veth2c1c3f6 table local proto kernel metric 0 pref medium
local fe80::3c02:e7ff:fe99:273e dev veth1bcf186 table local proto kernel metric 0 pref medium
local fe80::3c43:12ff:feb6:4407 dev vethe09ac37 table local proto kernel metric 0 pref medium
local fe80::58a3:30ff:feb0:8a2b dev vethe114d26 table local proto kernel metric 0 pref medium
local fe80::58bf:1eff:fe92:dbd2 dev veth67e374a table local proto kernel metric 0 pref medium
local fe80::8c92:c9ff:fe2f:c7ed dev veth0d84748 table local proto kernel metric 0 pref medium
local fe80::90ef:23ff:fe34:571c dev vethdb1c15b table local proto kernel metric 0 pref medium
local fe80::9400:ff:fe0d:bb91 dev eth0 table local proto kernel metric 0 pref medium
local fe80::a0fd:1eff:fe21:c662 dev veth0bf244b table local proto kernel metric 0 pref medium
local fe80::a42a:48ff:fe98:68ca dev vethd5f5a74 table local proto kernel metric 0 pref medium
local fe80::bceb:74ff:fe97:f466 dev veth33a8cad table local proto kernel metric 0 pref medium
local fe80::c811:f7ff:fefb:b7cc dev veth048588a table local proto kernel metric 0 pref medium
local fe80::cc10:61ff:fe25:571d dev veth5ae2bda table local proto kernel metric 0 pref medium
ff00::/8 dev eth0 table local metric 256 pref medium
ff00::/8 dev docker0 table local metric 256 linkdown pref medium
ff00::/8 dev br-cc7dcdf95b47 table local metric 256 pref medium
ff00::/8 dev veth048588a table local metric 256 pref medium
ff00::/8 dev veth33a8cad table local metric 256 pref medium
ff00::/8 dev vethe09ac37 table local metric 256 pref medium
ff00::/8 dev veth1bcf186 table local metric 256 pref medium
ff00::/8 dev veth2c1c3f6 table local metric 256 pref medium
ff00::/8 dev veth5ae2bda table local metric 256 pref medium
ff00::/8 dev veth67e374a table local metric 256 pref medium
ff00::/8 dev vethb29c88d table local metric 256 pref medium
ff00::/8 dev veth0d84748 table local metric 256 pref medium
ff00::/8 dev vethdb1c15b table local metric 256 pref medium
ff00::/8 dev vethe114d26 table local metric 256 pref medium
ff00::/8 dev veth0bf244b table local metric 256 pref medium
ff00::/8 dev vethdd92ee9 table local metric 256 pref medium
ff00::/8 dev vethd5f5a74 table local metric 256 pref medium

ip r get to 2a00:1450:4001:80b::200e:

2a00:1450:4001:80b::200e from :: via fe80::1 dev eth0 src fd4d:6169:6c63:6f77::1 metric 1024 pref medium

And ifconfig:

br-cc7dcdf95b47: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.22.1.1  netmask 255.255.255.0  broadcast 172.22.1.255
        inet6 fe80::42:47ff:fe7f:2c49  prefixlen 64  scopeid 0x20<link>
        inet6 fd4d:6169:6c63:6f77::1  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::1  prefixlen 64  scopeid 0x20<link>
        ether 02:42:47:7f:2c:49  txqueuelen 0  (Ethernet)
        RX packets 107906  bytes 13141154 (13.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 118687  bytes 221525604 (221.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::1  prefixlen 64  scopeid 0x20<link>
        ether 02:42:7a:b5:4f:c2  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 159.69.x.x  netmask 255.255.255.255  broadcast 159.69.20.27
        inet6 fe80::9400:ff:fe0d:bb91  prefixlen 64  scopeid 0x20<link>
        inet6 2a01:4f8:xxxx::1  prefixlen 64  scopeid 0x0<global>
        ether 96:00:00:0d:bb:91  txqueuelen 1000  (Ethernet)
        RX packets 1466656  bytes 2017338323 (2.0 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 242369  bytes 35789858 (35.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1557  bytes 150186 (150.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1557  bytes 150186 (150.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth048588a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c811:f7ff:fefb:b7cc  prefixlen 64  scopeid 0x20<link>
        ether ca:11:f7:fb:b7:cc  txqueuelen 0  (Ethernet)
        RX packets 28197  bytes 3349225 (3.3 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 26703  bytes 3201108 (3.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth0bf244b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::a0fd:1eff:fe21:c662  prefixlen 64  scopeid 0x20<link>
        ether a2:fd:1e:21:c6:62  txqueuelen 0  (Ethernet)
        RX packets 100933  bytes 15862061 (15.8 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 111009  bytes 11633473 (11.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth0d84748: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::8c92:c9ff:fe2f:c7ed  prefixlen 64  scopeid 0x20<link>
        ether 8e:92:c9:2f:c7:ed  txqueuelen 0  (Ethernet)
        RX packets 103300  bytes 13898479 (13.8 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 121634  bytes 12670159 (12.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth1bcf186: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::3c02:e7ff:fe99:273e  prefixlen 64  scopeid 0x20<link>
        ether 3e:02:e7:99:27:3e  txqueuelen 0  (Ethernet)
        RX packets 36  bytes 2696 (2.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1210  bytes 84788 (84.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth2c1c3f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::140f:77ff:fe9b:888  prefixlen 64  scopeid 0x20<link>
        ether 16:0f:77:9b:08:88  txqueuelen 0  (Ethernet)
        RX packets 222  bytes 595112 (595.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1392  bytes 97629 (97.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth33a8cad: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::bceb:74ff:fe97:f466  prefixlen 64  scopeid 0x20<link>
        ether be:eb:74:97:f4:66  txqueuelen 0  (Ethernet)
        RX packets 117683  bytes 10479133 (10.4 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 175621  bytes 14606191 (14.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth5ae2bda: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::cc10:61ff:fe25:571d  prefixlen 64  scopeid 0x20<link>
        ether ce:10:61:25:57:1d  txqueuelen 0  (Ethernet)
        RX packets 144626  bytes 14669024 (14.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 126561  bytes 17294944 (17.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth67e374a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::58bf:1eff:fe92:dbd2  prefixlen 64  scopeid 0x20<link>
        ether 5a:bf:1e:92:db:d2  txqueuelen 0  (Ethernet)
        RX packets 35  bytes 2626 (2.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1173  bytes 81306 (81.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethb29c88d: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fa:2aff:fe49:e066  prefixlen 64  scopeid 0x20<link>
        ether 02:fa:2a:49:e0:66  txqueuelen 0  (Ethernet)
        RX packets 58194  bytes 7207407 (7.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 51512  bytes 8688896 (8.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethd5f5a74: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::a42a:48ff:fe98:68ca  prefixlen 64  scopeid 0x20<link>
        ether a6:2a:48:98:68:ca  txqueuelen 0  (Ethernet)
        RX packets 15188  bytes 2025159 (2.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13932  bytes 2746121 (2.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethdb1c15b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::90ef:23ff:fe34:571c  prefixlen 64  scopeid 0x20<link>
        ether 92:ef:23:34:57:1c  txqueuelen 0  (Ethernet)
        RX packets 560  bytes 62645 (62.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1517  bytes 296504 (296.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethdd92ee9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::43:2cff:fe5c:bb6b  prefixlen 64  scopeid 0x20<link>
        ether 02:43:2c:5c:bb:6b  txqueuelen 0  (Ethernet)
        RX packets 1942  bytes 136953 (136.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2134  bytes 136680 (136.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethe09ac37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::3c43:12ff:feb6:4407  prefixlen 64  scopeid 0x20<link>
        ether 3e:43:12:b6:44:07  txqueuelen 0  (Ethernet)
        RX packets 8695  bytes 489502 (489.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20816  bytes 203318137 (203.3 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethe114d26: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::58a3:30ff:feb0:8a2b  prefixlen 64  scopeid 0x20<link>
        ether 5a:a3:30:b0:8a:2b  txqueuelen 0  (Ethernet)
        RX packets 210436  bytes 18913898 (18.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 160172  bytes 22027812 (22.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

And sysctl -A | grep forwarding | grep ipv6:

net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.mc_forwarding = 0
net.ipv6.conf.br-cc7dcdf95b47.forwarding = 1
net.ipv6.conf.br-cc7dcdf95b47.mc_forwarding = 0
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.default.mc_forwarding = 0
net.ipv6.conf.docker0.forwarding = 1
net.ipv6.conf.docker0.mc_forwarding = 0
net.ipv6.conf.eth0.forwarding = 1
net.ipv6.conf.eth0.mc_forwarding = 0
net.ipv6.conf.lo.forwarding = 1
net.ipv6.conf.lo.mc_forwarding = 0
net.ipv6.conf.veth048588a.forwarding = 1
net.ipv6.conf.veth048588a.mc_forwarding = 0
net.ipv6.conf.veth0bf244b.forwarding = 1
net.ipv6.conf.veth0bf244b.mc_forwarding = 0
net.ipv6.conf.veth0d84748.forwarding = 1
net.ipv6.conf.veth0d84748.mc_forwarding = 0
net.ipv6.conf.veth1bcf186.forwarding = 1
net.ipv6.conf.veth1bcf186.mc_forwarding = 0
net.ipv6.conf.veth2c1c3f6.forwarding = 1
net.ipv6.conf.veth2c1c3f6.mc_forwarding = 0
net.ipv6.conf.veth33a8cad.forwarding = 1
net.ipv6.conf.veth33a8cad.mc_forwarding = 0
net.ipv6.conf.veth5ae2bda.forwarding = 1
net.ipv6.conf.veth5ae2bda.mc_forwarding = 0
net.ipv6.conf.veth67e374a.forwarding = 1
net.ipv6.conf.veth67e374a.mc_forwarding = 0
net.ipv6.conf.vethb29c88d.forwarding = 1
net.ipv6.conf.vethb29c88d.mc_forwarding = 0
net.ipv6.conf.vethd5f5a74.forwarding = 1
net.ipv6.conf.vethd5f5a74.mc_forwarding = 0
net.ipv6.conf.vethdb1c15b.forwarding = 1
net.ipv6.conf.vethdb1c15b.mc_forwarding = 0
net.ipv6.conf.vethdd92ee9.forwarding = 1
net.ipv6.conf.vethdd92ee9.mc_forwarding = 0
net.ipv6.conf.vethe09ac37.forwarding = 1
net.ipv6.conf.vethe09ac37.mc_forwarding = 0
net.ipv6.conf.vethe114d26.forwarding = 1
net.ipv6.conf.vethe114d26.mc_forwarding = 0

And traceroute6 google.com:

traceroute to  (2a00:1450:4001:80b::200e) from fd4d:6169:6c63:6f77::1, 30 hops max, 24 byte packets
 1  * * *
 2  * * *

ip6tables-save:

# Generated by ip6tables-save v1.6.1 on Tue Jul 31 19:50:43 2018
*security
:INPUT ACCEPT [28763:1962044]
:FORWARD ACCEPT [699928:73444337]
:OUTPUT ACCEPT [28076:1907468]
COMMIT
# Completed on Tue Jul 31 19:50:43 2018
# Generated by ip6tables-save v1.6.1 on Tue Jul 31 19:50:43 2018
*raw
:PREROUTING ACCEPT [708780:74194437]
:OUTPUT ACCEPT [28076:1907468]
COMMIT
# Completed on Tue Jul 31 19:50:43 2018
# Generated by ip6tables-save v1.6.1 on Tue Jul 31 19:50:43 2018
*mangle
:PREROUTING ACCEPT [708780:74194437]
:INPUT ACCEPT [28763:1962044]
:FORWARD ACCEPT [699928:73444337]
:OUTPUT ACCEPT [28076:1907468]
:POSTROUTING ACCEPT [728004:75351805]
COMMIT
# Completed on Tue Jul 31 19:50:43 2018
# Generated by ip6tables-save v1.6.1 on Tue Jul 31 19:50:43 2018
*nat
:PREROUTING ACCEPT [74820:6308358]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [37:3024]
:POSTROUTING ACCEPT [35:2848]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s fd4d:6169:6c63:6f77::/64 ! -o br-cc7dcdf95b47 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::d/128 -d fd4d:6169:6c63:6f77::d/128 -p tcp -m tcp --dport 443 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::d/128 -d fd4d:6169:6c63:6f77::d/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 25 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::9/128 -d fd4d:6169:6c63:6f77::9/128 -p tcp -m tcp --dport 110 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::9/128 -d fd4d:6169:6c63:6f77::9/128 -p tcp -m tcp --dport 143 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::9/128 -d fd4d:6169:6c63:6f77::9/128 -p tcp -m tcp --dport 4190 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::9/128 -d fd4d:6169:6c63:6f77::9/128 -p tcp -m tcp --dport 993 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::9/128 -d fd4d:6169:6c63:6f77::9/128 -p tcp -m tcp --dport 995 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 465 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 587 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 443 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::f/128 -d fd4d:6169:6c63:6f77::f/128 -p tcp -m tcp --dport 110 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::f/128 -d fd4d:6169:6c63:6f77::f/128 -p tcp -m tcp --dport 143 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::f/128 -d fd4d:6169:6c63:6f77::f/128 -p tcp -m tcp --dport 4190 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::f/128 -d fd4d:6169:6c63:6f77::f/128 -p tcp -m tcp --dport 993 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::f/128 -d fd4d:6169:6c63:6f77::f/128 -p tcp -m tcp --dport 995 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::6/128 -d fd4d:6169:6c63:6f77::6/128 -p tcp -m tcp --dport 587 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::6/128 -d fd4d:6169:6c63:6f77::6/128 -p tcp -m tcp --dport 25 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::6/128 -d fd4d:6169:6c63:6f77::6/128 -p tcp -m tcp --dport 465 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::/64 ! -o br-35b96e790911 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::3/128 -d fd4d:6169:6c63:6f77::3/128 -p tcp -m tcp --dport 587 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::3/128 -d fd4d:6169:6c63:6f77::3/128 -p tcp -m tcp --dport 25 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::3/128 -d fd4d:6169:6c63:6f77::3/128 -p tcp -m tcp --dport 465 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 4190 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 993 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 995 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 110 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::5/128 -d fd4d:6169:6c63:6f77::5/128 -p tcp -m tcp --dport 143 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::/64 ! -o br-5e20ca02384a -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::/64 ! -o br-2f9f6d9c18d5 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 4190 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 993 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 995 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 110 -j MASQUERADE
-A POSTROUTING -s fd4d:6169:6c63:6f77::8/128 -d fd4d:6169:6c63:6f77::8/128 -p tcp -m tcp --dport 143 -j MASQUERADE
-A DOCKER -i br-cc7dcdf95b47 -j RETURN
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 443 -j DNAT --to-destination [fd4d:6169:6c63:6f77::d]:443
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 80 -j DNAT --to-destination [fd4d:6169:6c63:6f77::d]:80
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 25 -j DNAT --to-destination [fd4d:6169:6c63:6f77::5]:25
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 465 -j DNAT --to-destination [fd4d:6169:6c63:6f77::5]:465
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 587 -j DNAT --to-destination [fd4d:6169:6c63:6f77::5]:587
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 110 -j DNAT --to-destination [fd4d:6169:6c63:6f77::9]:110
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 143 -j DNAT --to-destination [fd4d:6169:6c63:6f77::9]:143
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 4190 -j DNAT --to-destination [fd4d:6169:6c63:6f77::9]:4190
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 993 -j DNAT --to-destination [fd4d:6169:6c63:6f77::9]:993
-A DOCKER ! -i br-cc7dcdf95b47 -p tcp -m tcp --dport 995 -j DNAT --to-destination [fd4d:6169:6c63:6f77::9]:995
COMMIT
# Completed on Tue Jul 31 19:50:43 2018
# Generated by ip6tables-save v1.6.1 on Tue Jul 31 19:50:43 2018
*filter
:INPUT ACCEPT [27576:1886276]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [26902:1813448]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
:MAILCOW - [0:0]
-A INPUT -j MAILCOW
-A FORWARD -j MAILCOW
-A FORWARD -o br-cc7dcdf95b47 -j DOCKER
-A FORWARD -o br-cc7dcdf95b47 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-cc7dcdf95b47 ! -o br-cc7dcdf95b47 -j ACCEPT
-A FORWARD -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -j ACCEPT
-A FORWARD -o br-35b96e790911 -j DOCKER
-A FORWARD -o br-35b96e790911 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-35b96e790911 ! -o br-35b96e790911 -j ACCEPT
-A FORWARD -i br-35b96e790911 -o br-35b96e790911 -j ACCEPT
-A FORWARD -o br-5e20ca02384a -j DOCKER
-A FORWARD -o br-5e20ca02384a -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-5e20ca02384a ! -o br-5e20ca02384a -j ACCEPT
-A FORWARD -i br-5e20ca02384a -o br-5e20ca02384a -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o br-2f9f6d9c18d5 -j DOCKER
-A FORWARD -o br-2f9f6d9c18d5 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-2f9f6d9c18d5 ! -o br-2f9f6d9c18d5 -j ACCEPT
-A FORWARD -i br-2f9f6d9c18d5 -o br-2f9f6d9c18d5 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::d/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 443 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::d/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::5/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 25 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::5/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 465 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::5/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 587 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::9/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 110 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::9/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 143 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::9/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 4190 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::9/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 993 -j ACCEPT
-A DOCKER -d fd4d:6169:6c63:6f77::9/128 ! -i br-cc7dcdf95b47 -o br-cc7dcdf95b47 -p tcp -m tcp --dport 995 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT
# Completed on Tue Jul 31 19:50:43 2018

brctl show:

bridge name     bridge id               STP enabled     interfaces
br-cc7dcdf95b47         8000.0242477f2c49       no      veth048588a
                                                        veth0bf244b
                                                        [all the other veth* interfaces - post is getting too long]
docker0         8000.02427ab54fc2       no
Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • The portion of your routing table that you posted looks OK. Though you really should post the complete routing table. Also, try `traceroute6 www.google.com`. – Michael Hampton Jul 30 '18 at 15:39
  • I updated the question, though I doubt that the lines that I omitted in the routing table are of any relevance to the problem... –  Jul 31 '18 at 13:29
  • You're right, they don't appear to be. Though it was impossible to be sure without looking. So, it looks like the system is trying to route IPv6 out the ULA network, which isn't what is in your plain routing table. I suspect you've got some additional rules somewhere. Please post the output of these commands: `ip rule` and `ip -6 r s table all` and `ip r get to 2a00:1450:4001:80b::200e` and `ip6tables-save` – Michael Hampton Jul 31 '18 at 13:42
  • Congrats, you've got something really weird going on here, and I don't know what it is offhand. I can see it appears to be selecting the wrong source address, and that's probably why routing is going haywire, but I'm not sure why it selected the wrong source address. One last thing, can you post the output of `brctl show`? – Michael Hampton Jul 31 '18 at 18:15
  • Thanks, I know it's weird... ;) By the way, it's a fresh installation, basically all I did was set up docker and run my app. – chindocaine Jul 31 '18 at 18:38
  • it would be easier to ask your provider for another /64 subnet and assign that subnet to your network and add it to your docker/daemon.json ``` /etc/docker/daemon.json { "ipv6": true, "fixed-cidr-v6": "2001:db8::/64" } ``` the /64 subnet of your host and the /64 subnet of your container should be different so it can route, you could ask to have your /64 cut in half but it doesn't work as well. fe80 isn't the problem, it'll always exist for link-local, you need a global unicast address that is routed to your host's IPv6 address from upstream, works for me. – Jacob Evans Jul 31 '18 at 19:52

1 Answers1

-1

From a clean install, you simply need to tell docker to enable IPv6 and provide it with a Global Unicast IPv6 Subnet (/64 or larger). This subnet must be routed to your existing IPv6 host IP.

example from my lab:

/etc/docker/daemon.json

{
    "ipv6": true,
    "fixed-cidr-v6": "2001:470:X:X::/56"
}

and test it

docker run --rm  -it byrnedo/alpine-curl ipv6.icanhazip.com

2001:470:X:X:0:242:ac11:4

An you can have multiple links with the same link-local address

 ip addr | grep "fe80::1"
    inet6 fe80::1/64 scope link
    inet6 fe80::1/64 scope link

if none of this works for you, or your upstream cannot provide you with an additional subnet or routes and if no port conflicts, just start your app with --net="host" instead (see http://www.debug-all.com/?p=163 for more).

Jacob Evans
  • 7,636
  • 3
  • 25
  • 55