My situation is fairly simple and, I guess, it's encountered fairly often. I have several projects which all look pretty much the same - a single project consists of:
- Static content servers - there are a couple of decent CDN solutions that solve this problem
- Frontend HTTP servers - which should get incoming requests, distribute them across backend servers, get the answers and post it back to users
- Backend HTTP servers - scalable, properly distributed across the globe
- Database servers / NoSQL servers / whatever stores the data
The problem here is (2) - frontend servers. We're using multiple technologies to provide fault-tolerance and load-balancing with frontends, namely:
- DNS round-robin & DNS zone switching - not really that good at load balancing, very slow to recover after a failure (i.e. switching of DNS zones takes at least 15-20 minutes in practice even with 1-minute TTL)
- BGP switching - fairly expensive, takes lots of organizational efforts (i.e. getting at least /23 block of IP address), not really practical to switch between more than 2 paths
- BGP anycasts - basically, also fairly expensive and complex
- Switching between servers in the same datacenter using VRRP/CARP at the router - most viable alternative, but not really fault-tolerant due to a single datacenter being SPoF
Come to think of it, I thought that there must be a ready-made SaaS solution for this problem. What I dream of for our next project is to make it this way:
- Create backends, deploy them across the globe, have their IPs ready
- Buy a domain name
- Sign up for an external service that will:
- Provide me with IP(s) that I will point my domain name to
- Have a guaranteed web servers listening at these IP(s) all the time
- Monitor my backends
- Forward requests that it gets to a least loaded working backend (ideally, adjusting load balancing policy using some geographical / network latency principles)
- Get response from a backend and send it back to the user
I did my homework and found out that there are several load balancing SaaS solutions available:
- Amazon CloudFront - but it's not really a frontend forwarder, but a frontend-serving-content (i.e. basically a CDN as a service)
- Amazon Elastic Load Balancing - seems to be what I was looking for, but it doesn't work with non EC2-hosted backends
- Azure seems to include some kind of "load balancer", but it also works only with Azure-hosted servers
- RackSpace Cloud Load Balancers also seems to work only with RackSpace-hosted backends
- Storm Cloud Load Balancer, again, seems to work only with Storm-hosted servers
I wonder if anything else exists that might make my dream deployment a reality? May be I'm wrong and such thing does not exist? Or I'm just searching for a wrong term and there's already another widely-accepted term for such a service?
