1

I'm trying to build an architecture for new k8s cluster with master HA on different sites (different subnets).

The issue I'm facing is finding an open source Software Load Balance (there is no physical on the site, and its an on prem env) who can support different subnets (GSLB).

The Load balancer is for the Apiserver and all connectivity to the masters.

Those anyone deploy that architecture? witch LB did you use? I cant use HAproxy or seesaw bc they dont support different subnets.

shirab
  • 11
  • 1
  • Welcome to the community! Could you please share some information about topology used: how are subnets connected to each other? What kind of routing is used? Are they located in one DC/on prem or different ones? What's the rtt between sites? Are there any NATs/VPNs? – moonkotte Jun 04 '21 at 09:52
  • Hi @moonkotte, Thank you for your answer. The environment is located in on prem site and its being routing as normal route (All connected to Core Router). So far we used HAProxy as our LB, but he's keepalive need L2 connection. I'm looking for a open-source LB for on-prem env that support LB, when the nodes are in different subnets. – shirab Jun 06 '21 at 06:09

1 Answers1

0

After some research and experiments, there are following considerations about one of solutions and its usage (this question can be resolved in different ways).

Nginx (regular, not PLUS) and haproxy can load balance traffic between different subnets for kubernetes API servers without any issues. However keepalived can't work in more than 1 subnet due to limitations in protocols such as HSRP, VRRP or CARP.

Potential solution here can be using architecture with at least 2 haproxy/nginx instances (for redundancy) within 1 subnet and keepalived + e.g. CARP layer on top of load balancers. This way health checking will be working on the top level and load balancers will have routes to other subnets with kubernetes api servers.

As for subnet availability and redundancy this can be solved using dynamic routing with a help of e.g. OSPF protocol as a part interior gateway protocol routing and BPG protocol for exterior gateway protocol routing.

Another solution which is open source but paid is Nginx PLUS. It can have active health checks (e.g. TCP active health checks) as well as GeoIP functionality.

Also Nginx PLUS supports GSLB. Here is a quote from Nginx official documentation:

Global server load balancing (GSLB) refers to the intelligent distribution of traffic across server resources located in multiple points of presence (PoPs). GSLB is most commonly implemented by controlling the responses to DNS requests, directing each user to the most appropriate destination IP address based on the availability, performance, and proximity of each PoP.

Which leads to using additional DNS service. E.g. NS1. There are some prerequisites to start with it:

  • A registered domain name
  • An NS1 account
  • Three or more deployed NGINX Plus instances, each with:

Please read a full description and setup of Nginx PLUS with GSLB article

Update:

GSLB concept is not only about load balancer and healthchecking (e.g. keepalived, but using DNS as well. Often using services offered by DNS providers are cheaper and more robust. For instance NS1 offers DNS based GSLB

Also you can get familiar with some of the current GSLB projects here - some of them a open source:

moonkotte
  • 290
  • 1
  • 8
  • Hello @shirab. Does this [answer your question](https://stackoverflow.com/help/someone-answers)? – Wytrzymały Wiktor Jun 14 '21 at 09:46
  • Not so much, the second part is not relevant for me- because Nginx PLUS is with price. The first one, still require that all my HAproxy will still be on the same subnet. In my architecture, it suppose to be 3 different subnet that contains a k8s master and a HAproxy server for LB- Which is not possible. Think about it as 3 different zones. I know F5 and Alteon and some other psychical LB can do it. But Im looking for free opensource that might help me. – shirab Jun 15 '21 at 09:57
  • I have extended an answer since GSLB is not only about using LBs and some healthchecking services. Also there is a list of open source projects. – moonkotte Jun 15 '21 at 12:24