0

kubernetes node status reports as below.

root@csp-nso-server78:~# kubectl get nodes 10.213.5.78
NAME          STATUS     AGE
10.213.5.78   NotReady   1h

The kubelet logs show below error

E0812 11:00:31.626169   10488 kubelet.go:2745] Error updating node status, will retry: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
E0812 11:00:31.696155   10488 kubelet.go:2745] Error updating node status, will retry: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
E0812 11:00:31.788017   10488 kubelet.go:2745] Error updating node status, will retry: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
E0812 11:00:31.897751   10488 kubelet.go:2745] Error updating node status, will retry: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
E0812 11:00:31.972326   10488 kubelet.go:2745] Error updating node status, will retry: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
E0812 11:00:31.972366   10488 kubelet.go:1183] Unable to update node status: update node status exceeds retry count

This seems to be something related to etcd. Can anyone help.

Akilesh
  • 131
  • 1
  • 4

1 Answers1

1

The error I have posted actually comes form kube-apiserver.

Apparently the kube-apiserver in my machine had been using an argument '--etcd-servers=http://x.x.x.x:14001'. In my setup ha proxy would listen on 14001 and then forward it to 4001 where etcd would listen.

Changing the init file for kube-apiserver to use port 4001 directly solved the issue in my case.

Still not sure how haproxy was hindering the communication but now things work as expected.

Akilesh
  • 131
  • 1
  • 4