18

I'm new to load balancing and I'm wondering if it's possible to use multiple load balancers to redirect traffic to my application servers. I don't really understand how this can be done. Shouldn't a domain name match one to one with a certain server's IP address (in this case the IP of one load balancer)? If each load balancing server has a different IP, how can the request be received by both load balancers (or by 10 load balancers or 50 or 100)?

uvsmtid
  • 847
  • 1
  • 6
  • 12
user3790827
  • 191
  • 1
  • 1
  • 6
  • Thank you for your response. So basically, if I want to use multiple load balancers to handle my traffic I only have to setup a different CNAME for each one of them? Specifically, if I need 10 load balancers to handle the traffic to my site, that's the only way to do it? – user3790827 Jul 12 '15 at 20:43
  • 1
    I recommend leaving questions open for at least a day before closing them. Even that is usually being hasty. Just because you've received an answer doesn't mean it's necessarily the only (or best) one, and marking your Q&A answered usually means it gets less attention. – Andrew B Jul 12 '15 at 22:16
  • 1
    @Anatoly I didn't make a decision yet. I've reviewed the solutions presented here and also talked with some of my friends who recommended me other solutions. I think that for my use case the best solution so far would be to use VPS servers from a cheap provider like DO or Vultr that don't offer Virtual IP and use the method used by Algolia with client load balancing. I need HA and scalability for the API only therefore there wouldn't be such a big deal if I create different subdomains for each load balancer. These end user of the widget will never notice them anyway. – user3790827 Jul 15 '15 at 15:53

2 Answers2

16

Using round robin DNS is not that great for high availability - if one server goes offline, clients will still try to connect to it and wait for a timeout.

There are other ways to achieve this.
1) Active/Passive load balancers
Basically one load balancer handles all traffic for one IP address.
If that balancer goes down, the passive node jumps in and takes over the IP.
Keep in mind that load balancers are pretty much only forwarding traffic, so for small to medium sized sites this can work out OK.

2) Active/Active load balancers
The same traffic IP is configured on both (or many more) load balancers.
Incoming traffic gets sent to all load balancers but a algorithm chooses which balancer should respond, all others discard that traffic.
Simple way to think of it, you have two load balancers:
When the requesting IP ends with an even number then load balancer A answers, otherwise load balancer B answers.

Of course your infrastructure must support this and there is overhead due to traffic getting sent but discarded.
More information, e.g. here: http://community.brocade.com/t5/SteelApp-Docs/Feature-Brief-Deep-dive-on-Multi-Hosted-IP-addresses-in-Stingray/ta-p/73867

faker
  • 17,326
  • 2
  • 60
  • 69
  • When you say "of course your infrastructure must support this" you mean I need an additional machine or VM that will send requests to the load balancers? – user3790827 Jul 13 '15 at 02:23
  • 2
    @user3790827 Infrastructure in this context is the network equipment, not servers.' – Jenny D Jul 13 '15 at 05:46
  • 1
    I'm planning use a cloud provider therefore I don't have direct control over the physical infrastructure. What should I ask my vps service provider for? – user3790827 Jul 13 '15 at 12:04
  • 1
    There is only abstract recommendations because it depends on a ton of details. We don't even know if it makes sense to have a multi hosted IP here - maybe his traffic is only a few hundred Mbit/s. If you need this, I would evaluate the proper software, check the requirements and figure out which provider supports it. Would DNS RR work? Sure. Would I use it? Depends on what kind of availability the owner of the business I'm working for is aiming for! – faker Jul 13 '15 at 13:59
  • @faker I'm sorry, I guess it's my fault because I didn't gave enough details. I want to build a javascript script that will be inserted into other people's websites and will gather traffic data( think Google Analytics), also it will access the server to display statistics for each page it's loaded on. Basically there would be a javascript file that will be loaded for each website it's used on. – user3790827 Jul 13 '15 at 14:31
  • Now solely for these js files maybe I can get away by using a amazon S3 and/or a CDN, but I would still have to do at least a GET request and a POST request for each page visited by the users of the sites this script is embedded in. – user3790827 Jul 13 '15 at 14:33
  • Obviously initially I probably wouldn't need more than a server but I would probably need it to have High Availability. But as you might imagine if this script would be embedded even on 100 sites or I would probably need to handle twice the number of page visits of those sites which can hundreds of millions. I would like to be able to handle millions of requests per second. – user3790827 Jul 13 '15 at 14:38
  • Hardware is also another problem. Initially I intended to use ARM servers from www.scaleway.com. But these servers have only 4 ARM cores and 2GB of RAM, you can't go beyond that in terms of hardware. Also the don't seem to offer Virtual IP's yet. The second and third options would have been vultr.com and digitalocean, even though I can get better hardware, the also don't offer Virtual IP's. Then there's AWS and Rackspace, they offer Virtual IP's, the offer various types of VPSs but they are quite expensive. – user3790827 Jul 13 '15 at 14:44
9

High Availability with load balancers is commonly implemented using a virtual ip address (VIP) protocol which allows several hosts (i.e. load balancers) to answer to one common ip address in one of several possible ways (variations on active/passive, active/active).

There are a good number of these protocols, the ones I have seen most with regular load balancers are VRRP and NLB (as well as a good many nondescript blackboxed protocols in appliances). Expanding to routers and firewalls one may also encounter CARP, HRSP, GLSP for instance.

This strategy has a number of benefits over DNS load balancing which is a simpler strategy (and which is taken care of in another answer).

DNS load balancing is burdened for instance with:

  • the slow turnover of dns caching mechanisms
  • limited load balancing algorithms (typically just round-robin)
  • the outsourcing of the load balancing decision to the client (through caching of the dns record)
  • Slow drain of service queues when a server (i.e. a load balancer) is taken out of rotation (based on dns record TTLs as handled by ISPs and clients)
  • Slow failover on load balancer failure

Using a virtual ip protocol for HA one may have a choice to achieve for instance:

  • Choice of load balancing algorithm amongst the load balancers
  • Server centric load balancing decisions (fascilitating for example service health based measures and routing)
  • Quicker drain of service queues when a load balancer is taken out of rotation.
  • Instant failover on load balancer failure

Only you know which strategy and protocol fits your scenario best.

ErikE
  • 4,676
  • 1
  • 19
  • 25
  • 1
    I'd also add that some load balancers support establishing BGP sessions with nearby routers, which allows you to set up [Anycast](https://en.wikipedia.org/wiki/Anycast) solutions. If the load balancer goes down or otherwise stops advertising for the VIP (failed health check), the next best routing candidate wins. The last sentence of this answer is imperative though: you really need to talk to your company's network admins. – Andrew B Jul 12 '15 at 22:29
  • Here is a nice description of what you describe in the first paragraph http://www.cisco.com/c/en/us/support/docs/application-networking-services/css-11000-series-content-services-switches/16556-vip-appguide.html – Martin Podval Nov 11 '15 at 10:23