Aim: to access apps that reside in a kubernetes (k8s) namespace using an HTTP(S) Load Balancer (HLB)
Methods
TCP load balancing (TLB) (layer 4)
- A k8s cluster has been created on Google Cloud Platform (GCP)
- A static IP was created
- A TLB was created, the static IP was assigned and the DNS has been updated
- Navigating to some.domain.com shows the app that has been deployed in the ‘some’ namespace
- another.domain.com shows the app that runs inside the ‘another’ namespace
- When additional namespaces are deployed, e.g. something-else then going to something-else.domain.com returns the app from that namespace as well.
HTTP(S) load balancing (layer 7)
- The assumption was that the apps could be accessed the same way, i.e. just replace the TLB with the layer 7 one.
- One of the official GCP was followed, backend was configured, default path was left as is and certificates were configured in the frontend section
- The DNS was updated
- After five minutes a HTTP 502 was returned when some.domain.com was consulted
- The ingress nginx log POD log was tailed, but no additional logging was added when the some.domain.com was refreshed
- The GCP logging indicated that there is a ‘backend error’
- The healthcheck of the loadbalancer was not working well initially and when a check 10256 HTTP was added the service became green
- When one navigates to some of the namespaces the following error is shown:
Error: Server Error
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.
Discussion
- Although the same health check, i.e. the one that is used for the TLB was added and turned the HLB to green, this could be a false positive.
- According to this Q&A, the issue is caused by an unhealthy service, but the TCP check 10256 was added, turned the service into green and then the issue persisted. Should another check by added?
- The assumption that the TLB would work the same looks incorrect. According to the GCP log the HLB cannot even route the traffic to a namespace.
Sub-questions
- What health check should be configured to check HLB?
- How does the routing of HLB deviate from TLB?