9

RFC 1034 requires us to assign at least two IP addresses for DNS servers. However, redundancy can already be achieved by a single IP address if we use anycast addressing. BGP anycast seems to scale well into hundreds or even thousands of servers.

If so, why do we still need multiple IP addresses for DNS servers? Does it actually enhance redundancy (contribute to availability) if we already have anycast in place, or is it just a myth?

What problems and errors can we expect to face if we only use a single IP address?

By that, I mean totally omitting secondary DNS addresses, or using a bogus IP (e.g. 1.2.3.4) for the second address when some setups require at least two.

Pacerier
  • 511
  • 15
  • 35

3 Answers3

16

A single anycast IP address does not give you the same redundancy as two unicast IP addresses in distinct IP prefixes would.

Often the hardest problem for redundancy is not when something fails completely, but rather when it is misbehaving just enough to still pass the health checks, but not actually be functional.

I have seen an anycast DNS setup where a DNS server went down, but packets would still get routed to that DNS server. Whatever was taking care of advertising the prefix might simply not be aware, that the DNS server had gone down.

It becomes even more tricky if the DNS server in question is not an authoritative DNS server, but rather a recursive resolver.

Such a recursive resolver would need to have both the anycast address for receiving queries from clients and unicast addresses for querying authoritative DNS servers. But if the unicast addresses went down, it could easily look healthy enough that it would still be routed queries.

Anycast is a great tool for scalability and reducing latency. But for redundancy it should not stand alone.

Multiple redundant anycast pools is however a good solution for availability. A well known example is 8.8.8.8 and 8.8.4.4. Both are anycast addresses, but they should never be routed to the same physical DNS server (assuming Google did their job well).

If you have 10 physical DNS servers, you could configure them as 2 pools with 5 servers in each pool or 5 pools with 2 in each pool. You want to avoid having one physical DNS server be in multiple pools simultaneously.

So how many IPs should you allocate? You need to have IPs that can be configured as anycast independently of each other. That usually means you'll need to allocate an entire /24 of IPv4 address space or /48 of IPv6 address space for each pool. This may very well limit the number of pools you can have.

Additionally if we are talking authoritative servers the DNS reply with all your NS records and A and AAAA glue should fit in a single 512 byte packet. For the root servers this worked out to 13 addresses. But that did not include glue and IPv6, so the number you'd reach would be lower.

Each pool should be as geographically distributed as possible. If you have 5 servers in Europe and 5 in Noth America and 2 anycast IPs, you do not create one pool spanning each continent. You put 2 from Europe in a pool with 3 from North America, and the other 5 in the other pool.

If you have more than 2 anycast pools, you can let a physical server temporarily be in more than one pool. But you should never allow a physical server to be in all pools at the same time.

Combining anycast and unicast is possible, but care must be taken. If you have IPs for two pools, I wouldn't combine. But if you only have a single anycast IP to work with, it may make sense to also include unicast IPs. The problem is that including unicast IPs will not give you as good latency and load balancing.

If a physical server is made available by both unicast and anycast, you may risk users reaching the same server as primary and secondary and lose access if it goes down. This can be avoided by only using unicast addresses of servers not in the anycast pool or by always providing users with two unicast addresses.

The more unicast addresses you put into the mix, the less queries will be sent to the anycast address, and the less benefit you will get from anycast in terms of latency and scalability.

kasperd
  • 29,894
  • 16
  • 72
  • 122
  • Do you mean that the best way to achieve scalability is to employ many secondary unicast addresses (a.k.a the old way of setting ns1.domain, ns2.d, ns3.d ... ns300.d) instead of using anycast? – Pacerier May 13 '14 at 09:07
  • @Pacerier No, that is not what I mean. I'll clarify that in my answer. – kasperd May 13 '14 at 09:10
  • +1. A routing error may even lead your unicast into hell (dead end). Having a second separate address means that you have more than one "ticket" in for that ;) – TomTom May 13 '14 at 09:18
  • @kasperd, Btw why do you say that using multiple unicast addresses does not achieve load balancing? Isn't it the case that the *load* is balanced out among this pool of unicast addresses? For example, the Asia client will be given the Asia server's unicast address and the US client will be given the US server's unicast address. – Pacerier May 13 '14 at 09:42
  • 2
    +1 I'd like to add that adding a bogus ip as second address sounds like a horrendous idea – Reaces May 13 '14 at 09:56
  • 1
    @Pacerier you do get load balancing from using multiple unicast addresses. The problem is the latency, clients simply don't know which IPs are nearby and which are far away. It also cannot scale as far. You only get to use about 5 servers, if you want A and AAAA glue records to fit in 512 bytes. Combining one anycast and multiple unicast is problematic for load balancing, as you are likely to get more load on the unicast addresses than on the anycast address. – kasperd May 13 '14 at 10:36
  • @kasperd, Would it be possible to dynamically change the 5 IP addresses in our DNS reply? For example, we can observe that the DNS request is coming from Asia via it's IP, then we customize a DNS reply to return the IP addresses of 5 of our Asia servers. If we see the DNS request coming from USA, we customize our DNS reply to return the IP addresses of 5 of our USA servers. Will this approach work? – Pacerier May 13 '14 at 10:56
  • @Pacerier those dynamic replies would have to happen at the level above you in the hierarchy. That most likely means you would need the authoritative DNS servers for the TLD to hand out dynamic replies. I don't know if any TLD supports that. If a TLD wanted to support that, the best way to do it would be for the TLD to be hosted on anycast IP addresses and have each physical server hand out a different set of NS records. – kasperd May 13 '14 at 11:17
4

Best practice is to use at least two addresses from different prefixes and giving them a name under two different TLDs. Both those addresses can be anycast if you want. Having only one IP address will give you a single point of failure. If the routing to that address doesn't work (configuration error, an anycast instance not working correctly, the prefix being hijacked etc) then your whole domain becomes unreachable.

Every anycast address will require at least a /24 IPv4 or a /48 IPv6 prefix to be routable in BGP. Smaller (longer) prefixes will usually not be accepted in the global routing table in many places.

Never ever put in a bogus IP address as a DNS server. It will cause severe delays for resolvers.

Sander Steffann
  • 7,572
  • 18
  • 29
  • Worse yet, that "bogus" IP address is somebody else's address. They would be receiving the queries. If they get annoyed with all of that traffic they could send replies with a high TTL to make it go away for a while. – kasperd May 13 '14 at 10:44
  • @kasperd, What about special *reserved* IP addresses which belongs to no one? – Pacerier May 13 '14 at 11:27
  • 1
    @Pacerier Using RFC 1918, 4193, or 6598 address space would limit the harm. But it would still cause resolution to be slowed down or even fail. – kasperd May 13 '14 at 12:48
3

The RFC 1034 only states that you require two DNS servers. This isn't a mandatory requirement, but a recommendation, so do with it what you will. Regardless, if you want HA, your 2 DNS servers can be assigned the same IP using anycast, and the only thing your end users would notice when one DNS server fails, is a momentary lack of connectivity as the network reconverges.

So in summary, yes using anycast is sufficient to be RFC 1034 compliant.

Reaces
  • 5,547
  • 4
  • 36
  • 46
  • Hmm, if a single IP address is all that's needed, why does Google offer two addresses (`8.8.8.8` and `8.8.4.4`) for their DNS servers? They already have anycast in place, so why not simply offer one address `8.8.8.8`? – Pacerier May 13 '14 at 08:58
  • 1
    At a guess I'd say to not interrupt connectivity during network convergence? Because they're a global player and want to make sure they're as broadly scoped as possible to eliminate any possible point of failure? I'm not entirely sure, but I do know we can't all be google :) – Reaces May 13 '14 at 09:01