0

If I run etcdctl it hangs. If I use --debug=true I see:

root@k8scp:~# kubectl exec -n kube-system -it etcd-k8scp sh


sh-5.1# ETCDCTL_API=3 etcdctl --debug=true endpoint health

{"level":"warn","ts":1643546720.7707205,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0005681c0/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection closed"}
127.0.0.1:2379 is unhealthy: failed to commit proposal: context deadline exceeded
Error: unhealthy cluster

I know the etcd is running fine.

Why does this command fail?

guettli
  • 3,113
  • 14
  • 59
  • 110

1 Answers1

1

I found the solution by looking at the logs:

root@k8scp:~# kubectl logs -n kube-system etcd-k8scp 

{"level":"warn","ts":"2022-01-30T12:45:09.762Z","caller":"embed/config_logging.go:169",
"msg":"rejected connection","remote-addr":"127.0.0.1:36846","server-name":"",
"error":"tls: first record does not look like a TLS handshake"}

The server wants TLS, but the default endpoint for etcdctl is localhost via http (not https).

guettli
  • 3,113
  • 14
  • 59
  • 110
  • If its in kubernetes, it most likely runs on TLS; you have to provide ca-cert, client-cert and client-key for the call to succeed, roughly `etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/etcd.pem --key=/etc/etcd/pki/etcd-key.pem member list` – P Marecki Jul 08 '22 at 15:26