90

I'd never heard of anycast until a few seconds ago when I read "What are some cool or useful server/networking tricks?".

The wikipedia "Anycast" article on it is quite formal and doesn't really evoke a mental picture of how it would be used.

Can someone explain in a few informal sentences what "anycast" is, how you configure it (just in a general sense), and what its benefits are (what does it make easier)?

username
  • 4,725
  • 18
  • 54
  • 78

8 Answers8

92

Anycast is networking technique where the same IP prefix is advertised from multiple locations. The network then decides which location to route a user request to, based on routing protocol costs and possibly the 'health' of the advertising servers.

There are several benefits to anycast. First, in steady state, users of an anycast service (DNS is an excellent example) will always connect to the 'closest' (from a routing protocol perspective) DNS server. This reduces latency, as well as providing a level of load-balancing (assuming that your consumers are evenly distributed around your network).

Another advantage is ease of configuration management. Rather than having to configure different DNS servers depending on where a server/workstation is deployed (Asia, America, Europe), you have one IP address that is configured in every location.

Depending on how anycast is implemented, it can also provide a level of high availability. If the advertisement of the anycast route is conditional on some sort of health check (e.g. a DNS query for a well known domain, in this example), then as soon as a server fails its route can be removed. Once the network reconverges, user requests will be seamlessly forwarded to the next closest instance of DNS, without the need for any manual intervention or reconfiguration.

A final advantage is that of horizontal scaling; if you find that one server is being overly loaded, simply deploy another one in a location that would allow it to take some proportion of the overloaded server's requests. Again, as no client configuration is required, this can be done very quickly.

Murali Suriar
  • 10,166
  • 8
  • 40
  • 62
  • Does anycast use more bandwidth compared to unicast? Imagine we have a thousand servers all sharing the same IP address, when we send a packet to *that* address, wouldn't it take up more bandwidth than compared to unicasting? – Pacerier May 13 '14 at 06:34
  • 6
    No - traffic sent to an anycast address will only arrive at one location advertising the address. You may be thinking of _multicast_, where packets sent to a multicast group address are sent to all hosts interested in that group. – Murali Suriar May 17 '14 at 10:10
  • 1
    Isn't the problem of "*Rather than having to configure different DNS servers depending on where a server/workstation is deployed*" solved by domain names? So what's the point of anycast when its simply duplicating what domain names are already doing? – Pacerier Sep 08 '17 at 05:26
  • 3
    @Pacerier - you can't use domain names to find your DNS server - that's a circular dependency. Example: configure dns.foo.com. as your DNS server. What IP address do I send requests to dns.foo.com to? I know, I'll look it up in DNS. etc. – Murali Suriar Sep 08 '17 at 10:20
34

One of the things that often confused me in understanding "anycast" is that, while it's a high level term, in practical implementation it usually boils down to two examples:

  1. Routers using BGP to advertise the same IP block via multiple AS paths as a rough way of directing users to a "closer" site. At the same time it provides for nearly transparent failover to the other sites by just retracting the routes from a troubled site. This can be useful for nearly any protocol, though obviously it raises lots of backend data synchronization concerns.

  2. Advertising the same service IP from multiple points within your own network (via static routing, OSPF, EIGRP, or whatever). If the routes are weighted differently it acts as a failover mechanism. If the routes are weighted evenly it can take advantage of the per-packet or per-flow load-balancing capabilities of most name-brand-vendor routers. You have to be careful that the application layer protocol is comfortable with this, that's why you almost always hear of it used with DNS where a request is always one packet and everything is stateless. Personally, I view this as a hacky intrusion of application layer concerns into the network layer when a combination of DNS and proper load-balancers will almost always be a better solution.

Greg
  • 167
  • 4
cagenut
  • 4,808
  • 2
  • 23
  • 27
  • 1
    +1 for mentioning one packet and statelessness – nponeccop Feb 17 '14 at 08:13
  • 3
    problem with DNS is that one can use google or some other remote resolver and nearest server to resolver might be different than to user. Of course this is valid only anycast is for non DNS usage (e.g. tcp anycast or other usage). – akostadinov Jul 11 '14 at 09:27
  • 1
    @cagenut, Would HTTP break in the case of per-packet load balancing (due to even weights or otherwise)? – Pacerier Apr 14 '16 at 21:03
17

Mainly used for UDP based services like DNS. Basically, you announce the same route out of multiple datacenters across the world. This way, your clients will be sent to the "best" and "closest" datacenter based on BGP routes. I put "best" and "closest" in quotes because network providers can play games and route traffic from certain networks differently. Generally, things work out for the best with anycast, but it's not a guarantee.

An example of this would be to list your DNS servers as 1.2.3.4 and 1.2.3.5. Your routers would announce a route for 1.2.3/24 out of multiple datacenters. If you're in Japan and have a datacenter there, chances are you'd end up there. If you're in the US, you'd be sent to your US datacenter. Again, it's based on BGP routing and not actual geographic routing, but that's usually how things break down.

diq
  • 710
  • 4
  • 9
  • 1
    How are conflicts avoided then? – Pacerier May 13 '14 at 06:36
  • I'm not sure what you mean by conflicts. BGP doesn't really allow for conflicts. It just picks a route. One (often ignored) problem with anycast is legal. Technically, anycasting is covered by different patents. No one enforces this, but different traffic types (CDN, DNS, TCP vs UDP) are covered by different patents. This paragraph should be covered with IANAL. – diq Jan 31 '16 at 21:01
  • So you mean that those ip2location services are inaccurate? Because there is no official location for an IP and it could be **anywhere** depending on which BGP computer you speak to? – Pacerier Mar 11 '18 at 14:15
9

Pursuant to my original response, I have just posted two more articles on my blog entitled: Anycast DNS - Part 3, Using RIP and Anycast DNS - Part 3, Using RIP (continued). The latter goes into more details, but at www.netlinxinc.com/netlinx-blog.html you will find actual recipes on how to configure Cisco routers and Open Source Quagga host-based routing software for Anycast DNS using RIP.

I am currently working on writing the fourth article in the series. This will provide recipies on how to deploy Anycast DNS using OSPF. Last in the series, I'll show recipes for deploying Anycast DNS using BGP.

Anycast DNS - Part 1, Overview

Anycast DNS - Part 2, Using Static Routes

Anycast DNS - Part 3, Using RIP

Anycast DNS - Part 3, Using RIP (continued)

netlinxman
  • 477
  • 1
  • 5
  • 10
4

Given this is mainly DNS at the moment...

Informally it makes your service more resilient and with better network access/latency/speed by allowing you to setup the same service in multiple locations across the world all using the same address. When someone queries for that address they are given the closest/best route.

From a server perspective:

If unicast is you going to a single person, and multicast is you going to several, and broadcast is you going to all people, then anycast is being schitzophrenic and having multiple personalities where the personality best suited to each person connects with them. Hmm. Not the best analogy.

Alex
  • 1,103
  • 6
  • 12
3

A really interesting use of anycast is DNS. You can place 5 different DNS servers in various physical and network locations but share a single (or sometimes both primary and secondary DNS) addresses. Depending on where the source is, they get routed to their closest node. This does some traffic balancing plus it provides redundancy if a DNS server dies.

dexedrine
  • 396
  • 1
  • 4
2

According to one of my colleagues, it's also useful as a DoS attack mitigation technique, as people can only attack the "nearest" anycast IP address, so if there are a lot of zombines in, say, the USA, your Euro site would be mostly unaffected, as they can't actually send packets to it.

Also it may be possible to use it as a way to (somewhat naively) filter spoofed packets if they're obviously coming from somewhere unlikely to be advertised in BGP as the correct route (e.g. packets coming into Europe when the ASN indicates a N American block).

MarkR
  • 2,898
  • 16
  • 13
  • 1
    Well, even in the simple case that would only stop the people who's AS path goes to that server---e.g. you may stop people who reach you from Comcast, but not AT&T. If you've got it setup "properly" for failover (track the L7 service, stop announcing when it goes down), then it's actually a lot trickier to handle DoS attacks since it'll nuke one server then move onto the next when it's announcements are taken down... – James Cape Nov 22 '10 at 23:18
  • @JamesCape, Interesting, But when they move to the next, the dos has failed since people are not able to connect right? – Pacerier Mar 11 '18 at 15:10
  • @Pacerier In the simplest case, where you just advertise the same IP everywhere with no intelligence on it, yes. However, if you try to get clever and withdraw the advertisement in the US because the service isn't responding, then the only advertisement left will be the one in Europe. So all the US zombies will then hit the next available server, and kill that one too. – James Cape Mar 13 '18 at 21:34
2

It is also good to note that Anycast is not good or reliable for some TCP connections that cannot survive resets or any long conversation.

Anycast IPs, using BGP, tell the internet that there are 2, 3 or more paths to a specific HOST, however in actuality these are NOT the same host, they are exact replicas of hosts advertised across multiple datacenters to achieve a lower latency connections.

For Example, I have 3 servers doing 301 non-www redirection for 198.251.86.133, if you ping this host, you may get DUPLICATE responses at times, or even drops depending where you are located, as my servers are US-East, US-West, and EUR. for short timed connections (like 301's which are browser cache'd) this give a fast response by a local server in the nearest datacenter.

From a redundancy standpoint there is none built into anycast, you would still need independent redundancy at each site as that IP will (in typical scenarios) always point to those datacenters.

Jacob Evans
  • 7,636
  • 3
  • 25
  • 55
  • Re "ping"; isn't ping a UDP connection? – Pacerier Mar 11 '18 at 15:14
  • Re "as that IP will (in typical scenarios) always point to those datacenters"; are you sure? If that server blackouts, they would start pointing to the next nearest server isn't it? – Pacerier Mar 11 '18 at 15:14