0

I'm running a Mastodon instance using docker-compose, which builds and runs several containers, that connect to each other through a Docker internal network (the relevant docker-compose.yml).

I'm running this on an up-to-date Debian 11 server. The issue is that with the default installation, the Docker containers cannot connect to each other ("No Route to Host"). One solution I found is to enable IP masquerade in the firewalld public zone: firewall-cmd --zone=public --add-masquerade, which solves the issue and allows the containers to talk to each other.

Sadly, this solution has a downside that I cannot easily live with: The same server has a second IP address on the same network interface, that I need to use for outgoing SMTP traffic generated by postfix. To make this work, I've set smtp_bind_address to the secondary IP address. This works perfectly fine until I enable IP masquerading with the above command. As soon as I do that, the source IP of all outgoing traffic from postfix gets changed to the primary IP, which breaks mail delivery.

A temporary fix I found is to reverse the address translation using a manual netfilter rule:

nft add chain nat postrouting { type nat hook postrouting priority 100 \; }
nft add rule nat postrouting oifname eth0 tcp dport 25 snat to <secondary ip>

Obviously, this is not a good idea if I'm using firewalld as well, also I assume this is not persisted in any way.

So, ideally there would be a way to make networking between the Docker containers possible without enabling masquerading (I don't understand why this is necessary anyway), so that I don't need to hack this "fix" into netfilter. If this is absolutely not possible, I'd also settle for a way to tell firewalld to add the equivalent of my manual netfilter rules. I assume there's a way to do that with direct rules, but I can't figure out the correct syntax for it.

Other firewall rules in place are mostly blocking and opening some ports to various services.

rnlf
  • 1
  • 1
  • *"The issue is that with the default installation, the Docker containers cannot connect to each other"* - Is that the case? That is really strange and something to look into rather than what you're doing/not doing with netfilter rules. https://docs.docker.com/compose/networking/ *"By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name."* – HBruijn Aug 08 '22 at 17:07
  • Yes, ideally, that's how it should work. I don't know if this is an issue with Docker on Debian 11, but without adding Masquerading, I can ping one container from another (so ICMP works), but they cannot connect via TCP (for example the container running the web interface cannot open a connection to PostgreSQL running in another container). Enabling masquerading solves that. I found this solution through another Serverfault question (https://unix.stackexchange.com/questions/199966/how-to-configure-centos-7-firewalld-to-allow-docker-containers-free-access-to-th) – rnlf Aug 08 '22 at 17:11
  • why not just use a second interface for the second ip? – djdomi Aug 08 '22 at 17:53
  • @djdomi, the server only has a single physical interface. All sources I found say "just add secondary IP to the physical interface". Honestly, I would be okay with using an additional virtual interface. I've looked for information on how to do that, but I have no idea how that works. Wouldn't it also ultimate route through the same physical interface and thus undergo masquerading as well? – rnlf Aug 08 '22 at 18:13

0 Answers0