I am only making my first steps into k8s land so please bare with me...
I am trying to bring up a small k8s cluster with Singularity as the containers runtime. I am following this procedure.
The problem is that the coredns pods fail to start, because of the following:
Jan 27 07:18:15 cent8ws sycri[1302]: #011Error: rpc error: code = Internal desc = could not set up pod network interface: error getting ClusterInformation: Get "https://[10.96.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
I googled up this problem and found that it was raised many times, but none of the suggested soloutions worked for me. I'm not sure if it is related to the fact that I am using Singularity or not... Please see some details about my system and the status of the ks cluster below.
Any help will be much appreciated.
Many thanks,
Oren
root@cent8ws ~]# cat /etc/redhat-release
CentOS Linux release 8.3.2011
[root@cent8ws ~]# rpm -qa | grep -e kub -e sing
webrtc-audio-processing-0.3-9.el8.x86_64
kubelet-1.20.2-0.x86_64
kubectl-1.20.2-0.x86_64
kubernetes-cni-0.8.7-0.x86_64
singularity-3.7.0-1.el8.x86_64
kubeadm-1.20.2-0.x86_64
kubeadm init --pod-network-cidr=10.0.1.0/24 --cri-socket unix:///var/run/singularity.sock --ignore-preflight-errors=All --upload-certs --node-name=$HOSTNAME
Jan 27 07:18:15 cent8ws sycri[1302]: #011Error: rpc error: code = Internal desc = could not set up pod network interface: error getting ClusterInformation: Get "https://[10.96.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
[root@cent8ws ~]# kubectl get pods --namespace=kube-system
NAME READY STATUS RESTARTS AGE
coredns-74ff55c5b-dldl2 0/1 ContainerCreating 0 5m50s
coredns-74ff55c5b-tbhkw 0/1 ContainerCreating 0 5m50s
etcd-cent8ws.localdomain 1/1 Running 0 5m52s
kube-apiserver-cent8ws.localdomain 1/1 Running 0 5m51s
kube-controller-manager-cent8ws.localdomain 1/1 Running 0 5m51s
kube-proxy-wb62q 1/1 Running 0 5m50s
kube-scheduler-cent8ws.localdomain 1/1 Running 0 5m52s
root@cent8ws ~]# kubectl describe pods coredns-74ff55c5b-tbhkw --namespace=kube-system
Name: coredns-74ff55c5b-tbhkw
Namespace: kube-system
Priority: 2000000000
Priority Class Name: system-cluster-critical
Node: cent8ws.localdomain/192.168.122.1
Start Time: Wed, 27 Jan 2021 07:17:47 +0200
Labels: k8s-app=kube-dns
pod-template-hash=74ff55c5b
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/coredns-74ff55c5b
Containers:
coredns:
Container ID:
Image: k8s.gcr.io/coredns:1.7.0
Image ID:
Ports: 53/UDP, 53/TCP, 9153/TCP
Host Ports: 0/UDP, 0/TCP, 0/TCP
Args:
-conf
/etc/coredns/Corefile
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Limits:
memory: 170Mi
Requests:
cpu: 100m
memory: 70Mi
Liveness: http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
Readiness: http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/etc/coredns from config-volume (ro)
/var/run/secrets/kubernetes.io/serviceaccount from coredns-token-29225 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: coredns
Optional: false
coredns-token-29225:
Type: Secret (a volume populated by a Secret)
SecretName: coredns-token-29225
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: CriticalAddonsOnly op=Exists
node-role.kubernetes.io/control-plane:NoSchedule
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m46s default-scheduler Successfully assigned kube-system/coredns-74ff55c5b-tbhkw to cent8ws.localdomain
Warning FailedCreatePodSandBox 70s (x26 over 6m45s) kubelet Failed to create pod sandbox: rpc error: code = Internal desc = could not set up pod network interface: error getting ClusterInformation: Get "https://[10.96.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
