28

Has anyone ever seen this before? Note that this happens not only with google.com, but with every domain I try. It's a wireless connection (WEP), but I'm not sure how that would be relevant:

$ curl -v google.com
# This takes about 60s to return
* getaddrinfo(3) failed for google.com:80
* Couldn't resolve host 'google.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'google.com'

$ wget google.com
--2011-11-28 14:44:08--  http://google.com/
Resolving google.com... failed: Name or service not known.
wget: unable to resolve host address `google.com'

$ ping google.com
PING google.com (209.85.148.147) 56(84) bytes of data.
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=2 ttl=54 time=136 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=3 ttl=54 time=34.0 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=4 ttl=54 time=34.3 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=5 ttl=54 time=42.5 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=6 ttl=54 time=44.7 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=7 ttl=54 time=34.5 ms
^C
--- google.com ping statistics ---
8 packets transmitted, 6 received, 25% packet loss, time 7007ms
rtt min/avg/max/mdev = 34.063/54.376/136.026/36.758 ms


$ host google.com
google.com has address 209.85.148.106
google.com has address 209.85.148.147
google.com has address 209.85.148.99
google.com has address 209.85.148.103
google.com has address 209.85.148.104
google.com has address 209.85.148.105
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.

$ host google.com 192.168.1.201
Using domain server:
Name: 192.168.1.201
Address: 192.168.1.201#53
Aliases: 

google.com has address 209.85.148.103
google.com has address 209.85.148.104
google.com has address 209.85.148.105
google.com has address 209.85.148.106
google.com has address 209.85.148.147
google.com has address 209.85.148.99
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.

$ cat /etc/resolv.conf 
# Generated by NetworkManager
nameserver 192.168.1.201

$ cat /etc/hosts
127.0.0.1       localhost
::1             localhost

$ netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.254   0.0.0.0         UG        0 0          0 wlan0
127.0.0.0       127.0.0.1       255.0.0.0       UG        0 0          0 lo
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 wlan0

Basically any application, including Firefox, can't work to do name lookups. What's more, if I take the wifi offline and plug in an ethernet cable, everything is fine.

Daniel Quinn
  • 555
  • 2
  • 6
  • 14
  • 1
    Maybe add some more info - is it just curl ? What about wget, browsers, ping etc. ? – Sandman4 Nov 28 '11 at 08:39
  • I see you marked an answer but what exactly was the issue and solution? Was it an SELinux problem? – Belmin Fernandez Nov 28 '11 at 17:10
  • The "solution" was just that the network appears to be right-gibbled. I'm not running any SELinux on the laptop and the "network" is just managed by a crappy store-bought wifi router. That answer was the one to help me figure out that I was dropping packets all over the place, so I figured it was something I couldn't solve and gave that guy the credit. Why, do you have a better idea? – Daniel Quinn Nov 29 '11 at 14:20
  • `ping` does call getaddrinfo with slightly different parameters https://github.com/iputils/iputils/blob/master/ping/ping.c#L574 `ai_protocol = IPPROTO_UDP` maybe that confuses getaddrinfo differnetly somehow? Seems the `host` command isn't always what's used either: https://unix.stackexchange.com/a/553438/8337 – rogerdpack Nov 21 '19 at 20:15

8 Answers8

10

Using this: https://www.centos.org/modules/newbb/viewtopic.php?topic_id=39343

I found a key command that helped me troubleshoot:

[root@localhost ~]# wget -6 URL Failed
[root@localhost ~]# wget -4 URL Worked

It's something to do with the default ipv6 stack that's causing problems with certain utils. Disable ipv6 to resolve.

Nicolas Pepinster
  • 103
  • 3
David T
  • 129
  • 1
  • 3
8

Check your /etc/nsswitch.conf. If the hosts line says something like

hosts:      files dns

I'm as confused as you. But if it says something like

hosts:      files

then the fact that DNS is working (see output of host command) won't help curl, which is doing name resolution via the standard OS libraries, which have been told not to use the DNS.

MadHatter
  • 78,442
  • 20
  • 178
  • 229
  • Hmm. I hadn't thought of that, but I just checked and it says `files dns` so I guess that's not it :-( – Daniel Quinn Nov 28 '11 at 08:49
  • 2
    In mine, there was a lot more stuff on the hosts line. "dns" was listed after "[NOTFOUND=return]" which, in my understanding, causes the system to return not found before even checking the configured DNS servers! Moving "dns" to before the notfound thing fixed my system (had to reboot). – trebormf Nov 29 '17 at 14:02
  • This helped me. Didn't have to reboot though. – DUO Labs Nov 11 '21 at 04:26
8

Perhaps you have some very weird and restrictive SELinux (or grsecurity...) rules in place?

If not, try strace -o /tmp/wtf -fF curl -v google.com and try to spot from /tmp/wtf output file what's going on.

Janne Pikkarainen
  • 31,454
  • 4
  • 56
  • 78
  • 1
    Looks like it's the wifi connection itself. The output file is FULL of stuff like this: `9344 poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)` – Daniel Quinn Nov 28 '11 at 10:44
  • @Daniel Quinn, can you post output of `/tmp/wtf`? – Sachin Divekar Nov 28 '11 at 10:44
  • Here's the output: http://pastebin.com/1y5Z48NK – Daniel Quinn Nov 28 '11 at 12:49
  • Hmm, looks like it's doing the query to 192.168.1.201. Can you do "host google.com 192.168.1.201" for sanity? Basically, DNS lookup specifically against your DNS server. – cjc Nov 28 '11 at 12:57
  • Done (added it to the question). And that works just fine -- which makes sense since issuing host without a server argument works correctly as well. It looks like it's just actual HTTP calls too, since ICMP (mostly, it drops the first ping) works. – Daniel Quinn Nov 28 '11 at 13:00
5

I had the same problem - host, nslookup resolves ok, curl - can't on the same hostnames.

After tcpdumping communication I found that curl tries to establish TCP (in addition to UDP) connection to DNS port, which was closed in my router. After tcp port 53 was enabled curl started to work flawlessly.

Another strange thing is that this problem does not surface if dns server is regular bind installation. If I use embedded into router DNS server, curl suddenly tries to use TCP ports even if it already received (!) answer via UDP 2ms before. I suppose this is bug.

Andrew
  • 51
  • 1
  • 1
2

There could be an error in your /etc/resolv.conf file that nslookup tolerates but curl does not.

The question asked was "How is it possible that I can do a host lookup but not a curl?"

This is possible because curl uses getaddrinfo() to resolve the FQDN, whereas nslookup does not. Instead, I believe nslookup parses /etc/resolv.conf using some other function or library, or via its own custom code. I did not look at source code to verify this, but you can prove it by adding whitespace in front of the nameserver token in /etc/resolv.conf. nslookup can parse this but getaddrinfo() cannot.


Example /etc/resolv.conf
 nameserver 8.8.8.8

If your resolv.conf has this error, or other errors that are tolerated by nslookup but not getaddrinfo(), then you can resolve an FQDN with nslookup, but you will not be able to use curl on that FQDN.

Fix: as root, edit /etc/resolv.conf and remove any leading whitespace on the nameserver line.

mrjmh
  • 121
  • 1
  • The `strace` output shows DNS queries being sent without receiving replies. That does not support the hypothesis of a parsing error on `/etc/resolv.conf`. However it is possible the recursor is faulty and using a different recursor might help. – kasperd Jan 18 '16 at 19:47
2

If this occurs for anyone trying to set up DNS for an AWS EC2 instance, make sure to also enable the IPv6 rules (::/0) for HTTP and HTTPS in the security group used by that instance.

David Schumann
  • 121
  • 3
1

I had this same problem on my VE (running on my laptop) today, and found it quite surprising. Dig and NSlookup works, but curl fails.

So for instance:

# curl -v google.com
* getaddrinfo(3) failed for google.com:80
* Couldn't resolve host 'google.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'google.com'

But when I saw David T's post here, I decided to try it with curl. So while this fails:

# curl  google.com -6
curl: (6) Couldn't resolve host 'google.com'

This succeeds:

# curl  google.com -4
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

The -6 specifies that curl uses IPv6 and the -4, to use IPv4. I get the same errors when using wget, so definitely some problems with the IPv6 stack on the Host.

All the other modifications to the nsswitch.conf file and other BIND conf files did not help as the problem is not with this utilities.

AkinO
  • 11
  • 1
0

Was your curl installation smooth? If possible try re-installing curl.

Try curl -v google.com to get more verbose output for debugging.

e.g.:

curl -v dnserror.test
* getaddrinfo(3) failed for dnserror.test:80
* Couldn't resolve host 'dnserror.test'
* Closing connection #0
curl: (6) Couldn't resolve host 'dnserror.test'

Are you getting similar output?

Sachin Divekar
  • 2,445
  • 2
  • 20
  • 23