Docker containers on multiple hosts with docker-compose

Question

I use docker-compose to run a small harem of containers in support of an application. The use of Docker's internal DNS resolver (e.g. db resolves to the database backend container) is key to these. I use a private bridged network created with docker network.

I am wondering if there is an easy and straightforward way, without the bureaucracy of Kubernetes and that, to take advantage of the DNS discovery mechanism while spreading the containers out onto multiple physical servers. One option is to simply build a separate Docker installation and private network network on another server and have these containers update DNS entries for themselves in my PowerDNS server upon startup, and there is an existing mechanism for that. However, I greatly prefer to leverage Docker's internal DNS discovery.

Any pointers would be appreciated!

Answer 1

I'd suggest you look into Docker Swarm, in my experience easier to set up. It should extend nicely from your current docker-compose.yml as well.

You should already have swarm if you run a recent docker-engine, you just need to enable it by running docker swarm init. You can try it out over at play-with-docker.

You might want to look at Bret Fisher's github for some samples. That repo is the accompanying material for the author's Udemy course which, imo, also gives a good overview.

Answer 2

Yeah, you can achieve this with Docker swarm. Docker swarm provides a solution to docker compose limitations with compelling features like:

• It uses Docker Engine CLI to create a swarm of Docker Engines where you can deploy application services without any additional orchestration tools. -Swarm is a cluster of Docker Engines.
• Swarm manager automatic scaling to achieve the desired state - easy scale up and scaling down
• Service discovery: Swarm manager nodes assign each service in the swarm a unique DNS name and load balances running containers. You can query every container running in the swarm through a DNS server embedded in the swarm.
• Rolling updates and security - Each node in the swarm enforce TLS mutual authentication and encryption to secure communications between itself and all other nodes.

As you can see, it has multi-node and master nodes HA support. To get started with Docker swarm, read Docker Swarm 101 guide. When confident with Docker Swarm, read Getting Started with Kubernetes guide to dive into Kubernetes