2

I have a StatefulSet (see https://gist.github.com/holyjak/fc6e35a4228d348cf89eef5303473cab#file-eks-pod-no-avail-volume-zone-yml) with a volumeClaimTemplate and a corresponding StorageClass definition. I run on AWS EKS.

I realized there was a problem as my pod was Pending with the warning "node(s) had no available volume zone". Looking into AWS EC2 console I noticed there was a Volume but it was not attached to the worker node, while kubectl get pv listed it as OK.

I manually deleted the Volume in EC2 and try to delete the PV as well:

$ kubectl delete pv <id>
persistentvolume "<id>" deleted

but this command, despite printing "deleted" hanged and get pv still showed the PV.

I have tried to delete also the StatefulSet but to no avail.

Jakub Holý
  • 363
  • 1
  • 3
  • 14

1 Answers1

4

Solution: I obviously had to manually delete also the persistent volume claim (kubectl delete pvc demo-db-storage-demo-db-deployment-0). After that also the (obviously pending) PV delete finished. I was able to apply by resources again and got a new Volume, PV, PVC, and a running pod.

Jakub Holý
  • 363
  • 1
  • 3
  • 14
  • Oddly enough, if you delete the pvc first (`demo-db-storage-demo-db-deployment-0` in this case), it will delete the underlying pv for you. It really should emit a message saying that it won't delete a resource that's still "in use" (claimed) and exit. – Andrew Beals Sep 15 '22 at 23:55