Right now I have
- cluster of 9 K8s server (1 master, 3 regular workers and 5 static content workers).
- Ingress is running as nodeport on 80 (I know this is not recommended).
- static.xxx.com DNS is pointing to static content workers.
I encountered issue with high networking traffic inside cluster. What happens?
- Request wants static content.
- It goes to the worker which can handle the request, however it still needs to be bootstrapped by the ingress, thus its redirected to the node which has nginx-ingres pod. (and it's not always on same server, its roundrobin policy by default)
- Request need to travel to ingress and back to any static content pod causing extra network issue (I might be mistaken in amount of jumps)
This staturates the link in the cluster. I have few ideas - have 2 ingresses in cluster (one for regular, and one for static content) and force static content ingress to hande request on same server. - change nginx-ingress policy to handle request on that server if possible.
How is it possible to achieve no jumps if request can be served on this host?
Is it possible to setup 2 ingresses in self managed cluster?
Is it possible to spawn nodeport only on certain hosts?