1

We are using external etcd cluster for k8s cluster. We connected master to this etcd server but receive

"tls: first record does not look like a TLS handshake"

How to fix this issues? (for eksctl side all is working correctly on etcd servers with same certificates)

ETCDCTL_API=3 /usr/local/bin/etcdctl member list   --endpoints=https://127.0.0.1:2379   --cacert=/etc/etcd/ca.crt   --cert=/etc/etcd/etcd-server.crt   --key=/etc/etcd/etcd-server.key
    b1fa8ebad0f4fa6, started, etcd-kube-cluster-1, https://10.105.113.*:2380, https://10.105.113.*:2379, false
    984a08591dda4911, started, etcd-kube-cluster-3, https://10.105.114.*:2380, https://10.105.114.*:2379, false
    b55b37a2544c7daa, started, etcd-kube-cluster-2, https://10.105.113.*:2380, https://10.105.113.*:2379, false

Kube-api server manifest updated with same certificates

Andrew Striletskyi
  • 25
  • 3
  • 1
    Have tried openssl to verify the certificate is correct. openssl s_client -showcerts -connect 127.0.0.1:2379 -cert /etc/etcd/etcd-server.crt -key /etc/etcd/etcd-server.key -CAfile /etc/etcd/ca.crt. Also can you please share me the etcd startup options and certificate details. – Kiruba Aug 07 '20 at 14:34
  • 1
    Mostly this issue could be the CA configured in the certificate. You are using endpoint in the command has 127.0.0.1, instead can you please try anything with any one of the client listen address like https://10.105.113.*:2379 – Kiruba Aug 07 '20 at 14:36
  • All is working with all options: [root@ip-10-105-113-108 kube-apiserver]# openssl s_client -showcerts -connect etcd-kube-cluster-2.test.com:2379 -cert etcd-client.pem -key etcd-client-key.pem -CAfile ca.pem CONNECTED(00000003) And etcdctl is working as well – Andrew Striletskyi Aug 09 '20 at 10:53

1 Answers1

0

The error message implies that ETCD server is rejecting your connection due to certificate or CN in URL is not valid for the certificate configured.

I guess the issue is with subjectAltName or CN configured in the etcd server, you could not have included 127.0.0.1 IP in the subjectAltName of etcd-server certificate. You can do any of the below 2 options to make it work.

  1. If you want to connect using 127.0.0.1 IP then this address need to added as subjectAltName in etcd server certificate.
  2. I presume you would have already added all you etcd server IPs as subjectAltName, so try connecting using etcd server IP/DNS name (instead of 127.0.0.1).

Thanks,

Kiruba
  • 116
  • 1
  • If I will execute this command from k8s master, It will return me correct values etcdctl --endpoints=https://etcd-kube-cluster-1.*.aws.in.*.com:2379 --cacert=ca.pem --cert=etcd-client.pem --key=etcd-client-key.pem member list -w table But kubelet can't authorize to etcd cluster and from etcd server I see TLS handshake errors – Andrew Striletskyi Aug 09 '20 at 12:04
  • Why you want to establish connection between kubelet and etcd , normally all interaction to etcd will be handled only by kube-api server. – Kiruba Aug 09 '20 at 13:08