I'm on Ubuntu 16.04, fully updated. Noticing some really really slow DNS resolution on things like curl/dns, and within python. I've done some searching, but nothing has solved it. I"m trying ot build an app that requires fast response time, but my app performance is getting crushed because of this issue.
I already fixed this once (or at least I thought I did) by turning off ipv6 which made my apt-get way faster, and that seemed to fix everything else for a while too-- but all the sudden, it's back to taking forever to resolve. Not sure what the deal is or if I updated some package and it reset it. wget/curl on a website takes forever to resolve, then all the sudden it does, and bam. I'm on a 100M/100M connection. I should mention that if I force --inet4-only with wget, it's instantatneous, making me think this is a ipv6 resolution thing again but I have verified that ipv6 is off:
http://www.neuraldump.com/2016/11/how-to-disable-ipv6-in-ubuntu-16-04-xenial-xerus/
Here's the first dump.
00:14:19.512317 IP 10.10.10.10.57771 > google-public-dns-a.google.com.domain: 27177+ A? www.huffingtonpost.com. (40)
00:14:19.512323 IP 10.10.10.10.57771 > google-public-dns-a.google.com.domain: 4945+ AAAA? www.huffingtonpost.com. (40)
00:14:19.516199 IP google-public-dns-a.google.com.domain > 10.10.10.10.57771: 4945 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., AAAA 2606:2800:221:18cd:1493:1b94:ea5:557 (154)
00:14:19.622120 IP resolver1-fs.opendns.com.domain > 10.10.10.10.57771: 27177 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., A 152.195.54.95 (142)
00:14:24.517261 IP 10.10.10.10.57771 > google-public-dns-a.google.com.domain: 27177+ A? www.huffingtonpost.com. (40)
00:14:29.522261 IP 10.10.10.10.57771 > google-public-dns-a.google.com.domain: 27177+ A? www.huffingtonpost.com. (40)
00:14:34.527351 IP 10.10.10.10.50963 > google-public-dns-a.google.com.domain: 44146+ A? www.huffingtonpost.com. (40)
00:14:34.529688 IP google-public-dns-a.google.com.domain > 10.10.10.10.50963: 44146 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., A 152.195.54.95 (142)
00:14:34.529730 IP 10.10.10.10.50963 > google-public-dns-a.google.com.domain: 8868+ AAAA? www.huffingtonpost.com. (40)
00:14:39.531792 IP 10.10.10.10.47933 > google-public-dns-a.google.com.domain: 44146+ A? www.huffingtonpost.com. (40)
00:14:39.629774 IP google-public-dns-a.google.com.domain > 10.10.10.10.47933: 44146 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., A 152.195.54.95 (142)
00:14:39.629841 IP 10.10.10.10.47451 > google-public-dns-a.google.com.domain: 8868+ AAAA? www.huffingtonpost.com. (40)
00:14:39.696671 IP google-public-dns-a.google.com.domain > 10.10.10.10.47451: 8868 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., AAAA 2606:2800:221:18cd:1493:1b94:ea5:557 (154)
Here's the second tcpdump. If I do it again immediately (it is the same no matter if it's a different domain name, I'm doing huffingtonpost again because of how the tcpdump filter I used is setup) the results are blazing fast:
00:15:09.575741 IP 10.10.10.10.47420 > google-public-dns-a.google.com.domain: 11015+ A? www.huffingtonpost.com. (40)
00:15:09.642151 IP google-public-dns-a.google.com.domain > 10.10.10.10.47420: 11015 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., A 152.195.54.95 (142)
00:15:09.642229 IP 10.10.10.10.57342 > google-public-dns-a.google.com.domain: 40060+ AAAA? www.huffingtonpost.com. (40)
00:15:09.708198 IP google-public-dns-a.google.com.domain > 10.10.10.10.57342: 40060 3/0/0 CNAME www.huffingtonpost.com.huffpost.akadns.net., CNAME cs593.adn.mucdn.net., AAAA 2606:2800:221:18cd:1493:1b94:ea5:557 (154)
The other weird thing is that if I do this within python (using requests library, and specifically, requests.packages.urllib3.connectionpool) it will do the normal 20-ish second stall on the first HTTP GET, but if I do a second one or any one after that, it's blazing fast just like it should be.
Below are two tcpdumnps of the 'slow' version ( you can see the 20s delay) and then again with what you'd expect. The opendns returning in the first one is very weird.
