Using SSH in check of Consul

Question

I have a server with a dockerized Consul Agent (CA). In this CA, I want to run a script from the host who checks how many disk space left in a service check.

For this, I created a service in the CA :

{
  "service": {
    "name": "disk-usage-var-re7", 
    "tags": [
      "type:check",
      "env:re7"
    ],
    "checks": [
      {
        "id": "disk-usage-var-re7",
        "name": "Disk Usage /var RE7",
        "args": [
          "sh",
          "-c",
          "ssh consul@hostserver 'disk_usage.sh /dev/mapper/centos-var'"
        ],
        "interval": "5s",
        "timeout": "30s"
      }
    ]
  }
}

But, when Consul runs him, I got this error : Host key verification failed.

I am using the official Docker image from Consul (Alpine), version 1.6.1, and I executed these commands in the container :

docker exec consul apk add openssh
docker exec -it consul ssh-keygen -t rsa

docker cp consul:/root/.ssh/id_rsa.pub /tmp

su consul
umask 077 && mkdir ~/.ssh
umask 077 && touch ~/.ssh/authorized_keys
cat /tmp/id_rsa.pub > /home/consul/.ssh/authorized_keys
rm -f /tmp/id_rsa.pub

When I try

docker exec -it consul ssh consul@hostserver 'disk_usage.sh /dev/mapper/centos-var'

The first time, SSH asks me :

The authenticity of host 'hostserver (::1)' can't be established.
ECDSA key fingerprint is SHA256:MlsT6sDr9xXuOurqBJu4e+a8m2De3Lu4ctJSB+5RNmk.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hostserver' (ECDSA) to the list of known hosts.

And after that, the script still works. But not with the Consul's check.

What am I doing wrong?

Many thanks.

score 0 · Answer 1 · answered Aug 24 '21 at 12:27

Ok, I found what happened. The default user in the container is root, but the user who starts Consul is consul.

So you have to run the container as the consul user specifying -u consul and use ssh-keygen with the correct user.

Now take a look at the correct commands :

docker exec -it -u consul consul ssh-keygen -t rsa

docker cp consul:/home/consul/.ssh/id_rsa.pub /tmp

su consul
cat /tmp/id_rsa.pub > /home/consul/.ssh/authorized_keys
rm -f /tmp/id_rsa.pub

And the script works !

Using SSH in check of Consul

1 Answers1