In a container cluster like Kubernetes or Docker 1.12 Swarm, how do you resolve with external DNS to the good container on the good host?

Question

Maybe I am dumb, maybe am I just not using the good words, but I can't find even one answer anywhere on Stackexchange or the whole Web.

I have created lots of website, mostly with Wordpress, Drupal, ...
(And recently, I have begun experiencing with Scala and Go, trying to create a website to gain some knowledge about these languages.)
My DNS provider is mostly DNSMadeEasy.

At first, I was using just one big dedicated server, under Debian, with my different websites just being in different folders, and Nginx, PHP-FPM, ... serving all my websites. Traditional setup.

Then few years ago, I switched to a Docker setup, still on one single host, with one Nginx container, one PHP-FPM container, one MySQL container, ... per website.
And one nginx reverse-proxy container in front of everything, to redirect the visitor of website example.com to the example.com containers.

Either with the traditional setup or the Docker-based setup, I was assigning the IP Address of my host to the A record of my different websites in DNSMadeEasy, and voilà, everything would work as expected.

NOW, I would like to switch from a single-host setup to several-host aka Cluster setup, using Docker Swarm (1.12 Swarm) or Kubernetes, to be able to scale more easily, as this was the purpose of switching to Docker in the first place.
As such, the containers for my different websites will be distributed accross the different hosts of my cluster.

I am planning to use a European cloud running on OpenStack, as I can't (<> privacy laws) and don't want to use (<> too expensive) American clouds like Google Cloud Engine or AWS EC2 (<> Patriot Act <> European Data), so please, don't tell me to use these clouds. I already have been able to setup a Kubernetes cluster, and a Docker Swarm cluster, in this Openstack cloud.

MY QUESTION : What I can't understand, is how do you point your websites DYNAMICALLY to the new services & pods which are automatically created ?

Let's say I have four hosts (1 master & 3 workers).
If I just do Round Robin DNS resolution, putting all the four IP of my four hosts in the A record of my websites so that it goes to any of the four, it will be a nightmare and a total mess, with the cache headers and everything. Moreover, with DNSMadeEasy, they have an API to automatically add and delete records, so I suppose it should be possible to automatically update the A record or the CNAME record of the websites like this plugin does with AWS Route53: https://github.com/wearemolecule/route53-kubernetes? (We can see that this use an external LoadBalancer so I suppose I need an external lb too for my setup, if I should follow that solution ?)

I am aware that both Kubernetes and Docker Swarm use DNS for service discovery, and etcd/consul for registration & election.
But from what I understand, this is used as an internal DNS System, not an external one, no ?
Or should I point the NS to my hosts and use my hosts instead of DnsMadeEasy !!??

Am I missing something ? Am I doing it wrong ?

Please someone help me, I begin to have huge headaches trying to sort things out :)

Y

Answer 1

Docker 1.12 with swarm-mode comes with built-in load balancing.

In your case, the primary benefit of that is that you don't have to dynamically update your DNS depending on what host a particular container ends up running on (which would most likely lead to disaster anyway, because of DNS ttl and caching).

Say for example you ran:

docker service create --name nginx -p 80:80 nginx

That will create a nginx service with one replica, so a single container will be started on a random host in your swarm. But the built-in load balancing will route requests on port 80 on any of the swarm hosts to the host where the container is run. And similarly if you scale the service with:

docker service scale nginx=2

That makes for a very simple setup:

1. Use the DNSMadeEasy round-robin feature and add A records for all the host IPs in your swarm.
2. Deploy your app with docker service create
3. Profit

From your question, it sounds like you'd like to run multiple sites on your swarm, eg. multiple apps all listening on port 80 and port 443. As @Flippy points out, to do that you currently have to run a layer 7 loadbalancer like HAProxy or nginx that inspects request host-headers and forwards requests appropriately. This is also simple to do with Docker 1.12 Swarm mode and this is a good getting started guide.

Answer 2

Kubernetes Built-in DNS

As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager cluster add-on. A DNS Pod and Service will be scheduled on the cluster, and the kubelets will be configured to tell individual containers to use the DNS Service’s IP to resolve DNS names.

Every Service defined in the cluster (including the DNS server itself) will be assigned a DNS name. By default, a client Pod’s DNS search list will include the Pod’s own namespace and the cluster’s default domain. This is best illustrated by example:

Assume a Service named foo in the Kubernetes namespace bar. A Pod running in namespace bar can look up this service by simply doing a DNS query for foo. A Pod running in namespace quux can look up this service by doing a DNS query for foo.bar.

For more information, check the Kubernetes Documentation.

NGINX Load Balancing

Default configurations for load balancing:

http {
    upstream myapp1 {
        server srv1.example.com;
        server srv2.example.com;
        server srv3.example.com;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://myapp1;
        }
    }
}

For more information, check the NGINX Documentation.

DNS REST API

You can then create a custom script to interact with your DNS provider API as you need.

For more information, check the DNSMadeEasy REST API Documentation.