2

I created 3 coreos machines at digitalocean using this cloud-config.yml: https://gist.github.com/socketwiz/d6fe23d19180a1ad8b5a where token was replaced with the one I retrieved from:

$ curl -w "\n" "https://discovery.etcd.io/new?size=3"

Up to this point everything works well. I can see the other machines, I can

$ curl -L http://<machine_ip>:2379/v2/keys/

on each one of them and get back valid data so I suspect my firewall settings are OK (but maybe not). My problem begins when I setup a docker container. First I get the IP address for docker0:

$ ifconfig docker0

Then I setup the docker container:

$ docker run -it alpine ash

Once that boots up I run:

$ apk update && apk add curl

Then I try to access etcd using the ip address I got from ifconfig:

$ curl -L http://172.17.42.1:2379/v2/keys/

But the curl command just hangs and eventually times out. I suspect I have something misconfigured in my cloud-config.yml either in the etcd2 section or in the iptables configuration, but I'm still kind of new to all of this and I've been tweaking around on it all day and just can't get past this problem. Any ideas would be much appreciated.

Ricky Nelson
  • 121
  • 2

1 Answers1

0

Discovering the IP etcd is listening on

The address etcd is listening on can be found in /run/systemd/system/etcd.service.d/20-cloudinit.conf, so a neat trick is to add some extra configuration to the etcd service so that it writes this to an environment file you can use in your etcd units.

This approach is useful when your cluster grows to a point where you want a separate etcd cluster to the actual workers - then you just have a global unit writing the environment file for each worker.

Creating /etc/etcd.environment

Let's assume you're running etcd on each machine

Firstly, add some extra configuration to the etcd service startup to create a new environment file containing the address ETCD is listening on. Here's a bash fragment which will write the necessary config for you...

cat << EOF > /run/systemd/system/etcd.service.d/30-environment.conf
[Service]
#write an environment file to use in other units
#see http://serverfault.com/a/741283/2411
ExecStartPost=/bin/bash -c "echo ETCD_ENDPOINT=\${ETCD_ADDR} > /etc/etcd.environment"
EOF
chmod 644 /run/systemd/system/etcd.service.d/30-environment.conf

You could reboot to have this take effect, or just reload like this...

systemctl daemon-reload
systemctl restart etcd

Using /etc/etcd.environment in a unit file

You can use this /etc/etcd.environment in your unit descriptions, and then pass the ETCD_ENDPOINT to your container so it sees it as an environment variable...

[Service]
EnvironmentFile=/etc/etcd.environment

ExecStart=/usr/bin/docker run --rm \
  -e ETCD_ENDPOINT=${ETCD_ENDPOINT} \
  --name example \
  ubuntu:15.10

Now your container can use that to build curl command lines, or better yet, copy the etcdctl binary into the container and use that.

Paul Dixon
  • 1,436
  • 3
  • 21
  • 35
  • ...and having written that, I realised you're using etcd2, and this is really etcd v1 specific :( You can take a similar approach but the variable names are different - I'll update this answer when I upgrade my clusters! – Paul Dixon Dec 07 '15 at 12:53
  • For giggles I just tried removing all of my firewall settings from my cloud-config.yml and its working. Now to figure out which line(s) is causing me grief. I'll post back here a working cloud-config when I figure it out. – Ricky Nelson Dec 08 '15 at 00:25
  • I'll admit I'm not a firewall expert and I can't come up with a config that works by commenting out any of the rules that begin with a -A. The only way I can get it to work is by commenting out the rules that begin with a : which seems all wrong. – Ricky Nelson Dec 08 '15 at 01:56