3

We have a step during in of our large map reduce that does DNS resolutions. The application itself uses the c-ares library as well as libcurl.

I notice that with each resolution, there is first an AAAA attempt, then immediately an A resolution. This is seen clearly with TCPdump both on our named server as well as on the host.

Here's a sample of traffic to S3

$ tcpdump dst port 53
20:28:35.259552 IP x.x.x.x.55271 > ec2-xx.amazonaws.com.domain: 21815+ AAAA? s3.amazonaws.com. (34)
20:28:35.261526 IP x.x.x.x.56058 > ec2-xx.amazonaws.com.domain: 21823+ A? s3.amazonaws.com. (34)

And it repeats for each resolution. Since we do more than 70 million resolutions during this process, it makes sense to want to cut out 50% of this as waste.

How can I disable IPv6 lookups in Ubuntu?

I've tried changing the precedence in /etc/gai.conf to:

precedence ::ffff:0:0/96 45

Yet I am still seeing ubuntu first try IPv6 resolutions. Any help would be greatly appreciated

Edit:

My /etc/resolv.conf is simple with just a nameserver set:

cat /etc/resolv.conf 
nameserver x.x.x.x
Nils Toedtmann
  • 3,202
  • 5
  • 25
  • 36
Stephen Wood
  • 133
  • 1
  • 5
  • Do you have `options inet6` set in your `/etc/resolv.conf`? – nickgrim Mar 01 '13 at 21:09
  • It's very simple, with just a nameserver declared. It's not symlinked with resolvconf either. – Stephen Wood Mar 01 '13 at 21:46
  • 3
    Sounds like you really need to do some local DNS caching. – Michael Hampton Mar 01 '13 at 23:20
  • I'm with Michael here. There is such a thing as overzealous optimization, and you'll need to make sure that this change is documented somewhere very clearly so that it is not forgotten down the road. – Andrew B Mar 01 '13 at 23:26
  • gai.conf controls the address used for the outgoing connection. It has nothing to do with name resolution (it is configured with IP addresses, not names, so it requires a AAAA and A request first). I agree with Andrew B and Michael Hampton that you need a local cache on your machine. – bortzmeyer Mar 03 '13 at 08:55
  • If they're happening that close together in time they're occurring in parallel. Are you sure you'll really save that much time by eliminating the extraneous lookup? (BTW, it's not really a BIND question -- your question more properly concerns the behavior of the client machine's resolver library.) – Michael McNally Mar 03 '13 at 22:51
  • I edited the title to reflect the fact that the accepted and only answer is specific to libcURL. – Nils Toedtmann Sep 29 '14 at 20:38

1 Answers1

3

IPv6, not IPv4, is the forward-looking version of IP, you know... IPv6 is sorry to see you go :-( But you can disable it.

If you are using libcURL then all you have to do is this:

curl_easy_setopt(easy_handle, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);

However, are you worried about the time these extra AAAA requests take, or the extra network traffic? I would hope that the A and AAAA are done in parallel so you shouldn't need to worry about a delay.

Celada
  • 6,060
  • 1
  • 20
  • 17
  • Thank you for this answer! Yes IPv6 is the future, but for this particular process it adds a lot of unnecessary overhead. The order of magnitude is quite staggering for the amount of resolutions. I'm hoping for a quick kernel hack I can do to prevent these outbound AAAA requests, but if I can't find one I'll try to implement this change with libcurl. – Stephen Wood Mar 01 '13 at 22:22
  • 1
    DNS resolution on Unix is done entirely in applications/libraries. You cannot have a "kernel hack" to configure it, the kernel knows nothing about the DNS. – bortzmeyer Mar 03 '13 at 08:53
  • This is the correct answer. I made your change in our library and it worked like a charm. – Stephen Wood Mar 05 '13 at 19:11