25

Our domain, grahamhancock.com is being wrongly resolved by a few people around the world, but it resolves correctly for most people.

When I run through a list of free open DNS providers, about 90% resolve correctly and give information consistent with our zone file. 10%, however, do not, and claim the IP address to be one linked to some Amazon EC2 instance which we've never owned or used ever in the past. Here are some example DNS servers giving the wrong information:

dig www.grahamhancock.com @173.84.127.88
dig www.grahamhancock.com @209.222.18.222

How could these servers have the wrong information, and how can we get back control of the situation?

Could this be something malicious, or a misconfiguration? We're a 1-million-hits-a-month site, with good search rankings, so we're probably a target for something malicious. The wrong IP address that the erroneous server are returning to some people points to some get-rich-quick site on an AWS EC2 instance.

What should we do?

Peter Mortensen
  • 2,319
  • 5
  • 23
  • 24
Duncan Marshall
  • 355
  • 1
  • 3
  • 7
  • 1
    Is that the actual domain name? – Drifter104 Jul 30 '15 at 14:28
  • 1
    Yes, that's the real domain. – Duncan Marshall Jul 30 '15 at 14:39
  • Some DNS server are badly configured too, the question is why your customer does not use their ISP DNS ? as atleast you can ask a hotline to get it fixed if on a ISP DNS. – yagmoth555 Jul 30 '15 at 14:45
  • Our users are using their ISP's DNS servers, but those servers won't accept my queries since I'm not their customer. The DNS servers above are public, and so I used them to test. If they're misconfigured, they're misconfigured in the same way, and have suddenly become misconfigured, since this wasn't happening yesterday. Here is the IP of one our user's ISP's DNS servers: 177.86.168.11. Our other users weren't responsive or proficient enough to give us their DNS server's IP. – Duncan Marshall Jul 30 '15 at 14:49
  • http://check-host.net/check-dns?host=grahamhancock.com – dmourati Aug 01 '15 at 04:11
  • 6
    May I in passing profoundly thank the original poster for including the actual domain name in his question, rather than redacting it? [As I have had occasion to note before](http://meta.serverfault.com/a/6574/55514), DNS questions are members of that class which are much easier to answer quickly and canonically when full dsclosure is made, and I personally think the very high quality of answers this question has had is partly due to many eyeballs being able to look directly at the problem. – MadHatter Aug 01 '15 at 06:32
  • @MadHatter Amen. One of those is [already in progress](http://serverfault.com/questions/710092/bind-9-2-server-refuses-to-resolve-cname-from-sub-zone). – Andrew B Aug 01 '15 at 06:38

2 Answers2

44

Drifter is correct, you have a nameserver configuration problem. Here's the tail end of the output from dig +trace +additional www.grahamhancock.com:

grahamhancock.com.      172800  IN      NS      ns1.grahamhancock.com.
grahamhancock.com.      172800  IN      NS      ns2.grahamhancock.com.
grahamhancock.com.      172800  IN      NS      server.grahamhancock.com.
ns1.grahamhancock.com.  172800  IN      A       199.168.117.67
ns2.grahamhancock.com.  172800  IN      A       199.168.117.67
server.grahamhancock.com. 172800 IN     A       199.168.117.67
;; Received 144 bytes from 192.35.51.30#53(f.gtld-servers.net) in 92 ms

www.grahamhancock.com.  14400   IN      CNAME   grahamhancock.com.
grahamhancock.com.      14400   IN      A       199.168.117.67
grahamhancock.com.      86400   IN      NS      ns2.grahamhancock.com.com.
grahamhancock.com.      86400   IN      NS      ns1.grahamhancock.com.com.
;; Received 123 bytes from 199.168.117.67#53(ns2.grahamhancock.com) in 17 ms

Your glue records are pointing to an IP address of 199.168.117.67, which returns the correct response. Your zone however is defining nameserver records ending in com.com. If we +trace one of those nameservers instead...

com.com.                172800  IN      NS      ns-180.awsdns-22.com.
com.com.                172800  IN      NS      ns-895.awsdns-47.net.
com.com.                172800  IN      NS      ns-1084.awsdns-07.org.
com.com.                172800  IN      NS      ns-2015.awsdns-59.co.uk.
;; Received 212 bytes from 192.26.92.30#53(c.gtld-servers.net) in 22 ms

ns1.grahamhancock.com.com. 30   IN      A       54.201.82.69
com.com.                172800  IN      NS      ns-1084.awsdns-07.org.
com.com.                172800  IN      NS      ns-180.awsdns-22.com.
com.com.                172800  IN      NS      ns-2015.awsdns-59.co.uk.
com.com.                172800  IN      NS      ns-895.awsdns-47.net.
;; Received 196 bytes from 205.251.195.127#53(ns-895.awsdns-47.net) in 16 ms

...we end up at someone's AWS hosted nameservers.

Your problem is something known as a glue record mismatch. Remote nameservers are initially learning about your domain via the glue records, but once those remote servers perform a refresh they end up querying the bogus nameservers that you've defined with an extra .com at the end.

This is not your only problem. You are listing the same IP address three times in your glue records, which is extremely volatile. You should always have multiple nameservers, they should never share a subnet or upstream network peer, and they should never be located at the same physical location. As matters currently stand, any brief routing problem between DNS servers and your single server will cause your domain to be temporarily unreachable.

Update:

This Q&A has been featured on the front page and is getting lots of comments. Unfortunately, that includes people who are just a little too eager to reply to this answer without checking to see if their points have already been addressed in the expanded comments.

The detail that most people seem to be overlooking is the comment that I'm quoting here:

  • [...] geo-redundant DNS servers prevent scenarios where a brief routing interruption results in temporary negative caching of nameservers. However brief the negative caching period ends up being, it will almost certainly exceed the amount of time that there was a connectivity interruption. [...] the number of scenarios where lack of DNS geo-redundancy won't create sporadic and difficult to troubleshoot availability problems is exactly zero.

If you think my understanding of negative caching of nameservers is wrong, that's open game for discussion, but outside of that you need to bring something to the table other than "it's a small site and who cares if both the website and DNS server are down at the same time". If you're saying this you don't understand the topic nearly as well as you think you do.

Second Update:

I went ahead and wrote a canonical Q&A that we can link to whenever the single DNS server topic comes up in the future. Hopefully this puts the matter to rest.

Community
  • 1
Andrew B
  • 31,858
  • 12
  • 90
  • 128
  • This is strange because the zone file is configured as it was out of the box by cPanel (according to our hosts). Also, when I examine the zone file in cPanel, there is no com.com problem. Everything is as it should be. As for the single nameserver problem, that's a financial decision that I have no control over. – Duncan Marshall Jul 30 '15 at 15:20
  • 15
    It doesn't matter what you see in the control panel, this is the reality of it. `dig @199.168.117.67 grahamhancock.com NS` makes this explicitly clear -- that data is coming from your servers. As for this being a "financial problem", I'm going to be blunt here: if you aren't going to run redundant DNS servers, you have **absolutely no business** operating your own DNS. You will have downtime. Unless you're very close to the owner, you will be responsible for that downtime and allowing this configuration to be implemented. – Andrew B Jul 30 '15 at 15:24
  • 1
    I hear you, but it's out of my hands. Anyway, thanks for the help. – Duncan Marshall Jul 30 '15 at 15:26
  • Sorry to be a hassle, but would you mind running that command again, and seeing if you're still getting the same .com.com problem? I'm trying, and it's not happening. Not sure if I just fixed it, or if I just can't replicate it from my location. – Duncan Marshall Jul 30 '15 at 16:02
  • 1
    It appears to be fixed now. – Andrew B Jul 30 '15 at 16:03
  • 1
    Since the nameserver is the same IP as the web server, a failing subnet, upstream network peer or burning building (same physical location) doesn't really matter here, because there is no use in having a backup nameserver telling you the IP of the unreachable web server. As soon as a second web server will be added, I guess they will also configure an NS on it so having some bit of redundancy then. – Bodo Thiesen Jul 30 '15 at 17:08
  • 1
    @Bodo Unless you have a copy of the zone file and have ruled out all possible services (not just www), that is not a safe assumption to make. – Andrew B Jul 30 '15 at 17:11
  • If money is the reason for not having a redundant name server (and Duncan already said so), then it is save to assume, that any service available on that domain is hosted on that single point of failure. I agree with your advice in general, having redundancy is always a good idea, but here there probably is just that single one lone server. – Bodo Thiesen Jul 30 '15 at 17:16
  • 2
    @Bodo Fair enough. That said, you're still overlooking the fact that geo-redundant DNS servers prevent scenarios where a brief routing interruption results in temporary negative caching of nameservers. However brief the negative caching period ends up being, it will almost certainly exceed the amount of time that there was a connectivity interruption. (or to put it more simply since I can't keep replying to this: *"blah blah blah DNS blah, the number of scenarios where lack of DNS geo-redundancy won't create sporadic and difficult to troubleshoot availability problems is exactly zero"*.) – Andrew B Jul 30 '15 at 18:44
  • 15
    You can get DNS hosting for $1 with redundant NS servers. It's not a financial choice. Don't be a cowboy. – JamesRyan Jul 30 '15 at 22:24
  • I didn't even know that registrars would *accept* a registration with all NS in the same C!? – Hagen von Eitzen Jul 31 '15 at 05:58
  • @AndrewB Surely the same thing applies to all servers? You will have downtime with web servers, mail servers, ... – user253751 Jul 31 '15 at 06:27
  • 1
    @immibis Webserver goes down, website goes down. DNS goes down, everything goes down. You also ignored the negative caching discussion completely. This will be my last comment on this answer, it's getting a bit silly. – Andrew B Jul 31 '15 at 07:03
  • 4
    There are plenty of *free* secondary DNS providers that are adequate for low-load zones. Or as @JamesRyan said above one can pay a (very small) amount of money for a professionally managed service with *some* sort of SLA. Both are viable alternatives for low-traffic sites. – user Jul 31 '15 at 09:49
  • @Andrew B, I'm not telling that having no secondary NS was a good idea, and actually I wouldn't want to use such a setup myself. Anyways, I actually didn't took stuff like negative response caching into account. For a non-profit size OTOH, financial reasons may in fact still justify skipping the secondary NS, because there you might better afford 5 minutes of users getting no access than paying one buck a month. Only a free secondary NS as suggested by Michael can then fully rule out the financial reasons. However: One server doesn't necessarily mean low traffic. – Bodo Thiesen Aug 01 '15 at 09:24
  • 1
    @JamesRyan Not being a cowboy, just that that isn't a decision I'm involved in, although I've relayed the advice. I didn't know that about the DNS hosting, and I'll suggest it. Are there companies that offer name servers without requiring they be your registrar? Any good names? – Duncan Marshall Aug 02 '15 at 03:55
11

Using the following tools gives a couple of clues

https://www.whatsmydns.net/#NS/grahamhancock.com reports that the NS records on the domain point to ns1.grahamhancock.com.com notice the extra .com

http://mxtoolbox.com/SuperTool.aspx?action=dns%3agrahamhancock.com&run=toolpage also reports that the same nameserver is reporting as authoritative.

If you take a look here http://www.dnsstuff.com/tools#dnsReport|type=domain&&value=grahamhancock.com it also reports that your names servers are open.

So it would appear somewhere along the line the nameservers are not set correctly. If they appear correctly to you through a control panel etc, you will need to speak to the provider so that they can check them on the actual servers.

Those links also have a full report on best practices and how to deal with them

Drifter104
  • 3,693
  • 2
  • 22
  • 39
  • 22
    And the parasites^H^H^H^H^H^H^H^H^Hpeople at `com.com` have configured wildcard DNS to take advantage of this kind of mistake. If your NS record points to anything.com.com they will answer whatever queries they get in a way that directs traffic to them. Try `dig @anything.com.com anyotherthing.com` and check out the authority and additional sections of the reply! –  Jul 30 '15 at 15:05