1

I have two node pools: NP1 and NP2. I have a StatefulSet with pods using ordinal index to retrieve their own configuration from external configuration source.

On certain conditions, I need to reschedule individual pods to either NP1 or NP2. Actual information about which pod should be run on which node pool is stored in a Consul KV object.

I can write a Kubernetes Operator which subscribes on that object and modifies StatefulSet in a way that it reschedules individual pods to a correct node pool when Consul KV object is updated. My question is how can I define different nodeAffinity for each pod?

I've seen some examples of using topologyKey with podAntiAffinity but in my case the decision which node pool a pod should be scheduled on is based on data completely external from kubernetes cluster.

Wytrzymały Wiktor
  • 731
  • 3
  • 10
Andrey Kuznetsov
  • 169
  • 2
  • 12
  • I'm pretty sure only a Mutating Webhook will allow you to do dynamic tricks like that; I think OPA has some concessions toward that use case, but I've thankfully never used it – mdaniel Dec 08 '21 at 16:33
  • Did you see [this page](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)? – Mikołaj Głodziak Dec 09 '21 at 12:58

1 Answers1

1

As far as I know, you can use mutating webhook as mdaniel has mentioned in the comment:

I'm pretty sure only a Mutating Webhook will allow you to do dynamic tricks like that; I think OPA has some concessions toward that use case.

You can also find here more information about Dynamic Admission Control.

Mikołaj Głodziak
  • 323
  • 1
  • 10