1

We are seeing the COREDNS pod is unable to lookup for primary DNS server which is resulting in readtimeout. Connectivity to primary DNS server work fine. Tried deploying Flannel/Calico/CANAL CNI but we still see the problem persist. nameresolution to kubdns and kubeapi server is working when i try to do a nslookup from a pod. We see ISTIO pod are unable to reach the below API server DNS address due to which pods are going to CrashLoopBackOff.

021-05-17T05:31:35.318792Z     error   k8s.io/client-go@v0.17.2/tools/cache/reflector.go:105: Failed to list *v1.Secret: Get https://10.96.0.1:443/api/v1/namespaces/istio-system/secrets?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout

Core DNS readiness/liveness probe failed
Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3

kubectl logs  coredns-775956cf78-zn9fr -n kube-system
.:53
[INFO] plugin/reload: Running configuration MD5 = 25c756253dfbdb23a23bab618a74ca05
CoreDNS-1.6.7
linux/amd64, go1.13.15, da7f65b
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0517 06:22:43.839379       1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.17.2/tools/cache/reflector.go:105 (started: 2021-05-17 06:22:13.838597206 +0000 UTC m=+0.029441714) (total time: 30.000641853s):
Trace[2019727887]: [30.000641853s] [30.000641853s] END
E0517 06:22:43.839413       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.2/tools/cache/reflector.go:105: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0517 06:22:43.839462       1 trace.go:116] Trace[1427131847]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.17.2/tools/cache/reflector.go:105 (started: 2021-05-17 06:22:13.83850082 +0000 UTC m=+0.029345241) (total time: 30.000767756s):
Trace[1427131847]: [30.000767756s] [30.000767756s] END



cat /run/flannel/subnet.env
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
cks cks
  • 11
  • 1
  • Hello @ckscks and welcome to Server Fault! How did you bootstrap you cluster? Did you restart your cluster after changing CNIs? – Wytrzymały Wiktor May 18 '21 at 07:39
  • Followed this document. https://docs.oracle.com/en/operating-systems/olcne/obe-deploy-olcne. Yes rebooted after CNI install. I have done couple of installs in other env already where i didnt see any issue! – cks cks May 18 '21 at 08:32
  • you need to check if there is any istio rule blocking the traffic. – c4f4t0r May 22 '21 at 21:03
  • @cks cks Has your issue resolved?If yes, can you post the procedure you've followed as Solution and accept it? – Fariya Rahmat Jul 01 '22 at 13:05

0 Answers0