After some research and experiments, there are following considerations about one of solutions and its usage (this question can be resolved in different ways).
Nginx
(regular, not PLUS) and haproxy
can load balance traffic between different subnets for kubernetes API servers without any issues. However keepalived
can't work in more than 1 subnet due to limitations in protocols such as HSRP, VRRP or CARP.
Potential solution here can be using architecture with at least 2 haproxy/nginx instances (for redundancy) within 1 subnet and keepalived
+ e.g. CARP layer on top of load balancers. This way health checking will be working on the top level and load balancers will have routes to other subnets with kubernetes api servers.
As for subnet availability and redundancy this can be solved using dynamic routing with a help of e.g. OSPF protocol as a part interior gateway protocol
routing and BPG protocol for exterior gateway protocol
routing.
Another solution which is open source but paid is Nginx PLUS. It can have active health checks (e.g. TCP active health checks) as well as GeoIP functionality.
Also Nginx PLUS supports GSLB. Here is a quote from Nginx official documentation:
Global server load balancing (GSLB) refers to the intelligent
distribution of traffic across server resources located in multiple
points of presence (PoPs). GSLB is most commonly implemented by
controlling the responses to DNS requests, directing each user to the
most appropriate destination IP address based on the availability,
performance, and proximity of each PoP.
Which leads to using additional DNS service. E.g. NS1.
There are some prerequisites to start with it:
- A registered domain name
- An NS1 account
- Three or more deployed NGINX Plus instances, each with:
Please read a full description and setup of Nginx PLUS with GSLB article
Update:
GSLB concept is not only about load balancer and healthchecking (e.g. keepalived
, but using DNS as well. Often using services offered by DNS providers are cheaper and more robust. For instance NS1 offers DNS based GSLB
Also you can get familiar with some of the current GSLB projects here - some of them a open source: