2

I have a Cookiecutter django project which uses Docker. I write my tests using pytest and I run my tests using docker-compose -f local.yml run django pytest. My local.yml file is in my root dir (where manage.py is)

I want to run these tests automatically with Gitlab CI. For that I set up my own gitlab server, installed and configured the runners. As executor I am using docker.

Then I tried to configure my gitlab-ci.yml file but I am not entirely sure how to do it correctly. This is what I have so far: 

image: docker

services:
 - docker:dind

stages:
 - test

tests-website:
 stage: test
 before_script:
   - apk update
   - apk upgrade
   - apk add python python-dev py-pip build-base libffi-dev openssl-dev libgcc
   - pip install docker-compose
 script:
   - docker-compose -f local.yml build
   - docker-compose -f local.yml run django pytest

 tags:
   - docker

My pipeline runs fine until docker-compose -f local.yml build. Then I get

couldn't connect to Docker daemon at http://docker:2375 - is it running? If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

I assume that this is a misleading error since I am struggling a bit to understand how I can execute my local.yml file. Or how I can run my tests in the pipeline. Maybe there are also more configurations I have to do to get the tests running? Or maybe what I have so far is completely wrong? I am not sure...

Can someone help me out with this one? Help is very much appreciated! Thanks so much in advance!

Micromegas
  • 231
  • 1
  • 10

1 Answers1

0

There are two potential problems:

  • First, docker:dind requires privileged mode
  • Second, since version 19.03 docker:dind listens on TLS socket only by default (port 2376).
/ # netstat -ntpul
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 :::2376                 :::*                    LISTEN      1/dockerd

So you can choose using TLS authentication and somehow pass certificate to inside a client container, or disable TLS doing something similar to:

tests-website:
 stage: test
 variables:
  DOCKER_TLS_CERTDIR: ""
 before_script:
   - apk update
   - apk upgrade
...

Then it degrades to plain TCP (port 2375)

/ # netstat -ntpul
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 :::2375                 :::*                    LISTEN      1/dockerd

What makes me curious, if you are not using kubernetes or something similar to run your jobs (plain docker I mean) then why not just use shell runner and talk to docker through the unix socket available on that machine? If it is not an option, then you can just pass machine's docker socket to inside a client container using mounts, rather than starting an additional instance of docker server

Владимир Тюхтин
  • 161
  • 1
  • Thank you so much for that great input. (I upvoted but missing the rep). I actually set the mode to privileged in my .toml file, so that should be ok. Setting up my server I created an ssl certificate. Do you mean I should pass this directory in the variable? I could also use a shell executor, that wouldn't be a problem (Not using kubernetes etc..) Do you think my script would run if I changed that? Or what would be different? Thanks again – Micromegas Oct 08 '19 at 13:37
  • And do you think the command ```docker-compose -f local.yml build``` can be used like I did in the script? – Micromegas Oct 08 '19 at 13:42
  • ohhhh I just tested it with ```DOCKER_TLS_CERTDIR: ""``` and it works!!! Fantastic, thank you so much! But for my better understanding: The cert would be my ssl cert? And would it be better with shell executor and what would I have to change? – Micromegas Oct 08 '19 at 13:56
  • As far as you have docker-compose installed on your runner machine alone with docker server you should be fine – Владимир Тюхтин Oct 08 '19 at 13:57
  • More about using docker:dind with TLS is here https://hub.docker.com/_/docker/ – Владимир Тюхтин Oct 08 '19 at 13:57
  • just in case someone wants to know how to configure TLS: https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03/ – Micromegas Oct 09 '19 at 08:35