0

In our company, we are using k8s in version 1.19 or older (have access to helm only). When we do deployment, there is a listener that will create some secrets that we need to mount in our pod. The thing is, such listener is doing his job in pretty late, and the deployment is scaled down to 0 after 10-15 minutes, because CrashLoopBackOff issue.

Is there any way to enforce deployment to keep trying initialize pod? I tried many things like increase livenessProbe and readinessProbe with adding failutreTreshold as 60 and period seconds as 30, so I think it should keep trying to check it for 30 * 60, but it doesn't.

Also I tried to set progressDeadlineSeconds to something like 3600 (one hour), but it doesn't work as well. No idea what to do.

psalkowski
  • 101
  • If secrets are missing, pod can't be created. If your Pod is in CrashLoopBackOff, the first thing to look at/share here would be a `kubectl logs -p`, and maybe a `kubectl describe pod `. There's no reason to change liveness, when we're not sure what's going on – SYN Aug 05 '22 at 18:50
  • @SYN yea, I know that and I know the reason why pod is in crashloop state. Also, crashloop is expected here, but secrets will be created by some listener within an hour. The problem is, deployment delete's my pod after 10-15 minutes and I want to avoid that. – psalkowski Aug 06 '22 at 19:03
  • "secrets will be created by some listener": that part makes no sense. If there's a Secret missing, then your Pod can't be created. Volumes that can't be attached would block your Pod somewhere in the containerCreating phase: it can't hit crashLoopBackOff. Please share logs and pod description/proof or didn't happen. – SYN Aug 06 '22 at 22:12
  • "deployment delete's my pod after 10-15 minutes". That part as well, is surprising. With OpenShift DeploymentConfigs, It would be normal. With Kubernetes Deployments: if my pod crashloopbackoff when I create my deployment: 6 months later, my restart counter would have gone through the roof, and that Pod would still be here. Your initial post just doesn't make any sense. Please update it with some actual data: pod definition, commands output. – SYN Aug 06 '22 at 22:19

0 Answers0