nginx in front of docker swarm services

Question

I am thinking about how this is meant to be done in a serious way for weeks now and didn't come to a conclusion yet. Maybe I think into the wrong direction.

Lets say you have a hundred web apps which come and go. You want an nginx configuration like that with the example of gitlab:

location / {
  proxy_pass http://gitlab;
}

As gitlab has been created with docker service create, nginx will be able to resolve gitlab by it's swarm-vip fake dns name within your ingress network.

BUT: only if the service container is running. Else, nginx will not start due to [emerg] 1#1: host not found in upstream "gitlab"

Now that's a neckbreaker when you have to run a high availability nginx and it's not your business to ensure that the proxy_pass'ed apps are running.

Everytime you update the nginx service, it will not come up if only one of the hundred other swarm services are not running even for the same second .. wtf?

If that ain't working, why in the world do we need name resolution in swarm? How do you solve this problem?

I thought about consul and dynamic generation of nginx virtual host templates (don't even think about docker-nginx-proxy!), but the apps are very different that you can say each app has its own individual configuration. And all this work not for a special reason, only to solve the resolving problem of nginx?

"BUT: only if the service container is running. Else, nginx will not start due to `[emerg] 1#1: host not found in upstream "gitlab"`": Could you elaborate? AFAIK, Nginx doesn't check if backends are running at startup. **Edit:** Ah, this is about DNS resolution, right? If so, one could use the IP. — gxx, Aug 15 '17 at 20:43
the IP of a container is changing every time a container restarts. One of the main reasons behind using swarm is the built-in name resolution. I am currently using IPs and this is bad. — sgohl, Aug 16 '17 at 19:10
The thing is, I did not find anything while looking for this problem. I can't be the only one having a need for that. Therefore I am almost sure I just oversee the right way to do it. Using IPs is definitely not. — sgohl, Aug 16 '17 at 19:12
nginx should just proceed logging this as a warning. Then I could take action with reload when a container rises — sgohl, Aug 16 '17 at 19:14
Hm, I see. Maybe `etcd` or `consul` or some such would work then. — gxx, Aug 16 '17 at 19:33
not really. I want the config files prepopulated and not dynamically created when the containers come up — sgohl, Aug 16 '17 at 20:04
Sounds hard to achieve, given the currently described environment. Can't help further, good luck! — gxx, Aug 16 '17 at 20:14

score 5 · Accepted Answer · answered Aug 20 '17 at 00:25

I'd recommend swapping out the statically defined nginx reverse proxy for traefik which is swarm aware and can dynamically reconfigure itself as services are deployed and destroyed.

Here's a sample implementation:

create a network for the traefik to talk to containers: docker network create proxy
Make a traefik.toml, here's an example:

traefik.toml

accessLogsFile = "/proc/1/fd/1"
defaultEntryPoints = ["http"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
[web]
address = ":8080"
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "localhost"
watch = true
swarmmode = true
constraints = ["tag==frontend"]

Sample compose file for traefik:

docker-compose.traefik.yml

version: '3'

networks:
  proxy:
    external:
      name: proxy

services:
  traefik:
    image: traefik:latest
    volumes:
    - ./traefik.toml:/etc/traefik/traefik.toml:ro
    - /var/run/docker.sock:/var/run/docker.sock
    ports:
    - 80:80
    - 8080:8080
    networks:
    - proxy
    restart: unless-stopped

Configure your app with labels and on the same network.

docker-compose.app.yml

version: '3'

networks:
  proxy:
    external: true

services:
  webtest:
    image: nginx:latest
    networks:
    - default
    - proxy
    labels:
    - traefik.frontend.rule=PathPrefixStrip:/webtest
    - traefik.port=80
    - traefik.docker.network=proxy
    - traefik.tags=frontend
    restart: unless-stopped

The rule above uses path prefixes for the container for simplicity, but you can use any rule you prefer to proxy your application.

now that's a suggestion. worth a try. I heard about traefik, well it's a new kid, but with your already provided examples - I'll give a try. more than I expected., Thanks a lot! — sgohl, Aug 21 '17 at 07:25
unfortunately, traefik is good in that particular point, but lacks a load of other features nginx has. traefik is no replacement to nginx. — sgohl, Oct 11 '17 at 12:01

nginx in front of docker swarm services

1 Answers1