1

I have setup a kubernetes cluster with CNI for networking, RBAC and TLS connections. I'm stuck since a couple of hours at getting my nodes ready.

root@master ~ # kubectl get nodes
NAME                STATUS     AGE       VERSION
us1-worker-node01   NotReady   3h        v1.6.1
us1-worker-node02   NotReady   3h        v1.6.1

The error seemed to come from kubelet

[...]
juin 14 13:54:03 us1-worker-node01 kubelet[19570]: I0614 13:54:03.192345   19570 manager.go:288] Starting recovery of all containers
juin 14 13:54:03 us1-worker-node01 kubelet[19570]: I0614 13:54:03.239022   19570 manager.go:293] Recovery completed
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.754864   19570 kubelet_node_status.go:230] Setting node annotation to enable volume controller attach/detach
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.760381   19570 kubelet_node_status.go:379] Recording NodeHasSufficientDisk event message for node us1-worker-node01
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.760566   19570 kubelet_node_status.go:379] Recording NodeHasSufficientMemory event message for node us1-worker-node01
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.760721   19570 kubelet_node_status.go:379] Recording NodeHasNoDiskPressure event message for node us1-worker-node01
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.760872   19570 kubelet_node_status.go:77] Attempting to register node us1-worker-node01
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.791650   19570 kubelet_node_status.go:128] Node us1-worker-node01 was previously registered
juin 14 13:54:02 us1-worker-node01 kubelet[19570]: I0614 13:54:02.791837   19570 kubelet_node_status.go:80] Successfully registered node us1-worker-node01
juin 14 13:54:08 us1-worker-node01 kubelet[19570]: I0614 13:54:08.171841   19570 kubelet.go:1808] SyncLoop (ADD, "api"): ""
juin 14 13:54:07 us1-worker-node01 kubelet[19570]: E0614 13:54:07.778441   19570 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR
juin 14 13:54:12 us1-worker-node01 kubelet[19570]: E0614 13:54:12.779166   19570 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR
juin 14 13:54:17 us1-worker-node01 kubelet[19570]: E0614 13:54:17.781032   19570 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR
juin 14 13:54:22 us1-worker-node01 kubelet[19570]: E0614 13:54:22.783057   19570 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR
juin 14 13:54:27 us1-worker-node01 kubelet[19570]: E0614 13:54:27.783449   19570 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR

The last error is repeating again and again...

here is the configuration of kubelet service

[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/kubelet \
  --allow-privileged=true \
  --cluster-dns=10.250.0.10 \
  --cluster-domain=cluster.local \
  --container-runtime=docker \
  --network-plugin=kubenet \
  --kubeconfig=/var/lib/kubelet/kubelet.kubeconfig \
  --serialize-image-pulls=false \
  --register-node=true \
  --require-kubeconfig \
  --tls-cert-file=/var/lib/kubelet/kubelet-client.pem \
  --tls-private-key-file=/var/lib/kubelet/kubelet-client.key \
  --v=2
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

CNI binaries are installed in /opt/cni/bin. Kubelet Subject Certificate is CN=system:node:us1-worker-node01, O=system:nodes, OU=Cluster

Any ideas ?

Shengis
  • 23
  • 1
  • 5

1 Answers1

1

OK...

The kube-controller-manager didn't had acces to the CA.key for the options : --service-account-private-key-file and --cluster-signing-key-file

With the good CA.key, no more problems !

Shengis
  • 23
  • 1
  • 5