6

I had an issue today where my deployment on my application servers couldn't be completed due to connection problems with github.

I noticed that when I perform a DNS lookup of github.com using google DNS servers i get the "correct/old" ip addresses which are also official documented by guthub HERE .

root@server# host github.com 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases: 

github.com has address 192.30.253.112
github.com has address 192.30.253.113
github.com mail is handled by 10 ALT4.ASPMX.L.GOOGLE.com.
github.com mail is handled by 10 ALT3.ASPMX.L.GOOGLE.com.
github.com mail is handled by 5 ALT1.ASPMX.L.GOOGLE.com.
github.com mail is handled by 1 ASPMX.L.GOOGLE.com.
github.com mail is handled by 5 ALT2.ASPMX.L.GOOGLE.com.

However, when i use my hosting provider DNS servers i get different results:

root@server# host github.com 213.133.98.98
Using domain server:
Name: 213.133.98.98
Address: 213.133.98.98#53
Aliases: 

github.com has address 18.195.85.27
github.com has address 35.159.8.160
github.com has address 18.194.104.89
github.com mail is handled by 1 ASPMX.L.GOOGLE.com.
github.com mail is handled by 10 ALT4.ASPMX.L.GOOGLE.com.
github.com mail is handled by 10 ALT3.ASPMX.L.GOOGLE.com.
github.com mail is handled by 5 ALT1.ASPMX.L.GOOGLE.com.
github.com mail is handled by 5 ALT2.ASPMX.L.GOOGLE.com.

When i contacted my provider as to why they are serving these results they stated that githubs a records have changed. Indeed that seems to be correct since when i perform a trace of githubs dns records using again google dns server i get the "new" records: 

root@server# dig github.com +trace @8.8.8.8
...    
text omitted
...
github.com.     172800  IN  NS  ns-520.awsdns-01.net.
github.com.     172800  IN  NS  ns-421.awsdns-52.com.
github.com.     172800  IN  NS  ns-1707.awsdns-21.co.uk.
github.com.     172800  IN  NS  ns-1283.awsdns-32.org.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20171215054800 20171208043800 11324 com. mBRl9D0i8jmeYbtZzR527TfVtbq2x6RSECv23chq0usVGZzVCQz5BYbV JaWeaQ1QWRuTWz3snYFkQBaG7SLQbipDEaVaMgjQ9qnHitJxwzEYPTn0 mT5nweDT+IVqP3NpppB748HAr9IiqqNOar1IyQokv3S59E9cK+s1W3V0 Mik=
4KB3QDAGSO6KO9JK2O5F2FO8F4C5FTA2.com. 86400 IN NSEC3 1 1 0 - 4KB4PTQQ5CTA7POCTGM7RUFC8B1RKTEU NS DS RRSIG
4KB3QDAGSO6KO9JK2O5F2FO8F4C5FTA2.com. 86400 IN RRSIG NSEC3 8 2 86400 20171212052031 20171205041031 11324 com. DmETcOQrFR+iFYhCH4xGJT+khPaTF4Ay50o+FrdpBvRTOPT9WTFf1wOF Ew3cQlBGHdwS2TiT+tLsUlshkmiKZpdH753Lac9Z0ZBU6fcB/PWOwMQX NGPWwYZFrGb8I2QsEvQreBM+WcftfdHGXHn5ziUx8phz1lbJuQXhVYyl LBk=
;; Received 840 bytes from 192.43.172.30#53(i.gtld-servers.net) in 17 ms

github.com.     60  IN  A   18.195.85.27
github.com.     60  IN  A   18.194.104.89
github.com.     60  IN  A   35.159.8.160
github.com.     900 IN  NS  ns-1283.awsdns-32.org.
github.com.     900 IN  NS  ns-1707.awsdns-21.co.uk.
github.com.     900 IN  NS  ns-421.awsdns-52.com.
github.com.     900 IN  NS  ns-520.awsdns-01.net.
github.com.     900 IN  NS  ns1.p16.dynect.net.
github.com.     900 IN  NS  ns2.p16.dynect.net.
github.com.     900 IN  NS  ns3.p16.dynect.net.
github.com.     900 IN  NS  ns4.p16.dynect.net.
;; Received 307 bytes from 205.251.198.171#53(ns-1707.awsdns-21.co.uk) in 12 ms

Why when I am queering for github.com using google DNS it retruns "old ips" but when i use a dns trace it returns the new ones? Does google dns returned cached records (even a day after)?

On the other hand, would github change their ip addresses without notifying anyone? Their documentation does not mention those "new" ip addresses and as far as i researched, whenever they made such a change, they also posted on their blog.

giomanda
  • 1,644
  • 4
  • 20
  • 30
  • 4
    Sometimes websites will do 'geographical load balancing' or 'DNS load balancing' where DNS servers will respond with different IP addresses based on the IP address of the requester. This enables traffic to be routed to datacenters/points-of-presence that are physically close to the requester. – alex.forencich Dec 08 '17 at 20:35
  • 1
    https://help.github.com/articles/about-github-s-ip-addresses/ takes you to the ranges that you should reasonable expect from them. – chicks Dec 12 '17 at 18:38

3 Answers3

9

If you must rely on IP addresses, and I can sympathize with your comment, you should refresh them regularly. GitHub publishes the IP addresses they use on https://api.github.com/meta

Dennis Kaarsemaker
  • 18,793
  • 2
  • 43
  • 69
5

Don't rely on IP addresses (it's not the '60s), they will change anytime the server changes provider, or implements a CDN, or a mirror, or a set of load balancers. Domain names are less likely to change.

And they don't have to notify anyone if they do anything on the backend, they can just mirror the infrastructure, change providers and change the domain records. Business as usual. This is trivial and sysadmins around the world are doing this all the time.

ThoriumBR
  • 5,272
  • 2
  • 23
  • 34
  • I dont really rely on ip addresses however iptables uses only ip address. Even if you pass a rule using a hostname it will resolve its ip and then add the rule with ip based. Output filter on servers is enabled hence why i lost connectivity with github – giomanda Dec 08 '17 at 13:21
  • 1
    There are lots of other ways to do outbound filtering with iptables. I've used rules to allow specific users outbound traffic, or to allow specific processes outbound traffic. At the end of the day, restricting outbound traffic by IP is going to cause this issue often enough to be annoying, but seldom enough that people forget how to fix it. – Harald Dec 12 '17 at 18:51
  • 1
    One option to deal with this is to use dnsmasq as a DNS resolver, get it to put results for github.com in an ipset, then reference the ipset in your iptables rule. Another option (arguably better, but more effort) is to use squid as an outbound proxy server - allow only the proxy server to access external sites then filter with squid ACLs by domain name rather than IP address. – Paul Gear Dec 12 '17 at 22:25
2

alex.forencich's comment on your question probably explains what you're seeing.

The A records for github.com may be different depending on where you're asking from. Keep in mind the "you" in the previous sentence really refers to whatever recursive DNS server your server's resolver is talking to. Your ISP's DNS server answers with IPs in AWS's eu-central-1 Availability Zone, which is in Frankfurt, Germany -- totally reasonable since your ISP is Hetzner.

I don't know Google's public recursive DNS servers work, but the answers you get when you ask there seem to be ones intended for North America.

Coming back to the reason you asked in the first place, if for some reason you need to know the IPs your internal hosts are going to be talking to, you'll need to look at what they use for their DNS servers, and use the same servers to grab the IPs for use on your firewall rules. There isn't a single "real" answer, so what you should be aiming for is consistency.

Drew Bloechl
  • 734
  • 3
  • 6