Curl local host names on Mac OS X Yosemite

Question

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names.

Set up a simple http server to test:

$ python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...

Now I can hit localhost:8000 in chrome. I can even wget it. But in curl, this happens:

$ curl localhost:8000
curl: (7) Failed to connect to localhost port 8000: Connection refused

However, this works:

$ curl 127.0.0.1:8000

I read this answer about wget proxy settings, but it didn't help, because this works:

$ wget --proxy=off localhost:8000

This is really frustrating, because I have a few different loopback hostnames listed in my /etc/hosts file so I can develop apps locally, and I'm used to debugging them with curl.

I've tried with the version of curl that ships with osx:

$ curl --version
curl 7.37.1 (x86_64-apple-darwin14.0) libcurl/7.37.1 SecureTransport zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz

$ curl localhost:8000
curl: (7) Failed to connect to localhost port 8000: Connection refused

$ curl 127.0.0.1 # works

And I've tried compiling curl with brew:

$ /usr/local/Cellar/curl/7.38.0/bin/curl --version
curl 7.38.0 (x86_64-apple-darwin14.0.0) libcurl/7.38.0 SecureTransport zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz

$ /usr/local/Cellar/curl/7.38.0/bin/curl localhost:8000
curl: (7) Failed to connect to localhost port 8000: Connection refused

$ /usr/local/Cellar/curl/7.38.0/bin/curl 127.0.0.1:8000 # works

Answer 1

I just got it to work by commenting out one of the IPv6 loopback lines from my /etc/hosts file:

#fe80::1%lo0    localhost

Now all of my loopback hostnames work, not just localhost. I wonder what's up with this?

Answer 2

Alternative (does not require sudo or modifying /etc/hosts) - always use ipv4 until curl gets smarter.

$ echo '--ipv4' >> ~/.curlrc

(then everything will work as desired)

Answer 3

First of all, 0.0.0.0 is a special address meaning "any IPv4 address".

A socket can be bound to either IPv4 or IPv6 protocol. If a socket is bound to 0.0.0.0, that means it will listen to any IPv4 trying to connect to it, and will be represented as it follows:

$ nc -l 0.0.0.0 8085
$ lsof -i4 -Pnl | grep 8085
  nc        23994 [xxx]    3u  IPv4 [xxx]      0t0  TCP *:8085 (LISTEN)

The * sign is equivalent to 0.0.0.0 on IPv4.

For IPv6:

$ nc -l :: 8085
$ lsof -i6 -Pnl | grep 8085
  nc        24145 [xxx]    3u  IPv6 [xxx]      0t0  TCP *:8085 (LISTEN)

The * sign is equivalent to :: on IPv6, as in the official specification.

The reason is that curl tries to resolve to a random localhost entry in /etc/hosts, and as @NickRetallack mentioned, that entry is the chosen one by curl when resolving localhost in its default mode (assumedly IPv6 or IPv4, whatever resolves first).

Forcing it in --ipv4 mode, as @CharlesHebdough suggested, will make curl resolve localhost to 127.0.0.1 (assuming there are no other IPv4 entries for localhost in /etc/hosts).

Each implementation will resolve localhost as they wish, hence why you had intermittent success with different tools.

To be the most precise possible, use 127.0.0.1 instead of localhost, but it will bound you to IPv4. localhost gives you the flexibility to work in both IPv6 and IPv4 protocols, however in some implementations you might have trouble, as in that specific version of curl.