Efficent networking in k8s - 2 ingress strategies

Question

Right now I have

1. cluster of 9 K8s server (1 master, 3 regular workers and 5 static content workers).
2. Ingress is running as nodeport on 80 (I know this is not recommended).
3. static.xxx.com DNS is pointing to static content workers.

I encountered issue with high networking traffic inside cluster. What happens?

1. Request wants static content.
2. It goes to the worker which can handle the request, however it still needs to be bootstrapped by the ingress, thus its redirected to the node which has nginx-ingres pod. (and it's not always on same server, its roundrobin policy by default)
3. Request need to travel to ingress and back to any static content pod causing extra network issue (I might be mistaken in amount of jumps)

This staturates the link in the cluster. I have few ideas - have 2 ingresses in cluster (one for regular, and one for static content) and force static content ingress to hande request on same server. - change nginx-ingress policy to handle request on that server if possible.

How is it possible to achieve no jumps if request can be served on this host?

Is it possible to setup 2 ingresses in self managed cluster?

Is it possible to spawn nodeport only on certain hosts?

Answer 1

How is it possible to achieve no jumps if request can be served on this host?

Set the Local value of service.spec.externalTrafficPolicy annotation in Ingress Controller's Service of NodePort. It will proxy the requests to local endpoints, never forwarding traffic to other nodes = packets sent to the nodes w/o specific endpoints (Pods) are dropped. Read here more on this native K8S Service feature.

Is it possible to setup 2 ingresses in self managed cluster?

Yes, it's.You just differentiate between them in Ingress resource definition with use of kubernetes.io/ingress.class annotation (e.g. kubernetes.io/ingress.class: "nginx").
Read here more about option of having deployed Multiple Ingress controllers in single K8S cluster.

Is it possible to spawn nodeport only on certain hosts?

Indirectly via usage of node affinity, which enforces Pods to be scheduled/run on specific Nodes. With aid of feature asked in your first question (service.spec.externalTrafficPolicy: Local) you will achieve it.