avahi: ping can't resolve hostname, but nslookup can

41

21

ping tells me that it can't resolve some hostname ("ping: unknown host domain.company.local") in a URL but when I use host or nslookup on the same computer on the command line, the resolutions works fine (i.e. it's fast and reliable).

What could be causing this?

More testing: Firefox, wget and ping have the same problem. Pinging the IP address works.

OS: Linux (Ubuntu 13.04)

EDIT My /etc/resolv.conf reads:

nameserver 127.0.1.1
search domain.company.local

netstat reports:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      -               

so something is running on this port (nslookup also reports it uses 127.0.1.1 as DNS server).

There is no /etc/*inetd.conf, so I'm not sure which application serves this port.

It seems that dnsmasq is used:

/usr/sbin/dnsmasq --no-resolv --keep-in-foreground --no-hosts --bind-interfaces
   --pid-file=/var/run/NetworkManager/dnsmasq.pid --listen-address=127.0.1.1
   --conf-file=/var/run/NetworkManager/dnsmasq.conf --cache-size=0 --proxy-dnssec
   --enable-dbus=org.freedesktop.NetworkManager.dnsmasq
   --conf-dir=/etc/NetworkManager/dnsmasq.d

All the config files and folders are empty. Since nslookup says it uses 127.0.1.1#53 my guess is that dnsmasq works even without a configuration. But how does it know which parent DNS to query?

EDIT2 Disabling dnsmasq as suggested by harrymc didn't help. So I ran strace ping which gave me this odd output (just the interesting parts):

open("/etc/host.conf", O_RDONLY|O_CLOEXEC) = 4
read(4, "127.0.0.1\tlocalhost\n#127.0.1.1\ta"..., 4096) = 613
...
open("/lib/libnss_mdns4_minimal.so.2", O_RDONLY|O_CLOEXEC) = 4
read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\f\0\0\0\0\0\0"..., 832) = 832
...
mmap(NULL, 2105560, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f7829b00000
...
socket(PF_FILE, SOCK_STREAM, 0)         = 4
fcntl(4, F_GETFD)                       = 0
fcntl(4, F_SETFD, FD_CLOEXEC)           = 0
connect(4, {sa_family=AF_FILE, path="/var/run/avahi-daemon/socket"}, 110) = 0
fcntl(4, F_GETFL)                       = 0x2 (flags O_RDWR)
fstat(4, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f782a4f8000
lseek(4, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
write(4, "RESOLVE-HOSTNAME-IPV4 domain.com"..., 44) = 44
read(4, "-15 Timeout reached\n", 4096)  = 20

So ping looks in /etc/hosts which makes sense. Then it loads and mmap()s /lib/libnss_mdns4_minimal.so.2 which makes sense as well.

But then it talks to avahi!?

Which led me to this forum post: ping doesn't make a dns request.

My /etc/nsswitch.conf also contains this line:

hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4

If I ping a working address, I see that the process also loads /lib/libnss_mdns4_minimal.so.2 but then, it does a DNS query via port 53.

So my guess is now that /lib/libnss_mdns4_minimal.so.2 is somehow noticing that the IP address ends with .local and not with .com and then the [NOTFOUND=return] is triggered.

How do I fix this?

Aaron Digulla

Posted 2014-01-21T11:14:17.430

Reputation: 6 035

What's in your /etc/resolv.conf? – Joseph R. – 2014-01-21T12:23:59.323

Which is correct and which is incorrect? Should the hostname resolve or shouldn't it? You didn't tell us which of two completely different problems you have. (And if it should resolve, explain in as much detail as possible how and why it should resolve, as that will likely lead to the explanation of why it doesn't.) – David Schwartz – 2014-01-21T12:43:14.953

... and what your HTTP proxy settings are for Chrome, Firefox, and wget. – JdeBP – 2014-01-21T13:27:14.743

@DavidSchwartz: I'm expecting resolution to work. What I don't understand how nslookup or host can resolve the name and anything else on the system can't. – Aaron Digulla – 2014-01-21T15:39:15.627

@JdeBP: I'm using a proxy.pac that returns DIRECT for this URL. But it never gets so far; all the programs already fail to resolve the host name (no matter whether I configure to use a proxy or not). – Aaron Digulla – 2014-01-21T15:40:13.117

@AaronDigulla What is different about this name compared to names that work? – David Schwartz – 2014-01-21T15:42:15.993

@DavidSchwartz: It's a local/internal name (ie. something that only our internal DNS server knows about). I could imaging that web browser use special DNS lookup code but how can the command line tools ping and nslookup disagree? – Aaron Digulla – 2014-01-21T15:59:41.137

Under Windows I know that nslookup opens an Internet connection on the DNS port and issues an external query, whereas ping uses the locally installed DNS client service, which from what you say appears to be dnsmasq. I don't know enough about Linux and dnsmasq, but evidently there is a problem with it. Why do you need dnsmasq? The Ubuntu documentation talks about possible conflicts when installing it.

– harrymc – 2014-01-23T20:04:47.843

Can programs like Chrome, ping, telnet and wget resolve any hostnames (e.g., Internet sites)? – Scott – 2014-01-24T00:30:11.127

@Scott: Yes. Most sites work, only this one doesn't. – Aaron Digulla – 2014-01-24T09:11:25.623

@harrymc: I didn't install it. I just installed Kubuntu. I was surprised to find dnsmasq on my system and I know nothing about it. On my system, NetworkManager starts the process. – Aaron Digulla – 2014-01-24T09:53:58.573

The linked doc specifically says the conflicts are with Network Manager, and recommends to install dnsmasq-base but not dnsmasq. You might try uninstalling it but keeping dnsmasq-base. – harrymc – 2014-01-24T10:09:28.103

@harrymc: That is already the case. I only have dnsmasq-base installed. – Aaron Digulla – 2014-01-24T10:17:04.453

1

If you are not doing connection sharing with other devices or VMs thru your computer, you could turn off dnsmasq in Network Manager. Edit /etc/NetworkManager/NetworkManager.conf and comment the dns=dnsmasq line (put a # in front of it) then do a sudo restart network-manager. That will turn off the local resolver. (source)

– harrymc – 2014-01-24T10:58:59.993

@harrymc: Thanks, that works as well but I like the "fix the AVAHI config" approach better. Still, turn it into answer and I'll upvote it. – Aaron Digulla – 2014-01-29T16:32:31.040

@AaronDigulla: Done. – harrymc – 2014-01-29T18:35:20.917

Answers

33

As described in detail in this blog post, you need to edit /etc/avahi/avahi-daemon.conf:

[server]
domain-name=.alocal

This binds the daemon to the domain .alocal instead of the default .local.

and restart the daemon with:

sudo service avahi-daemon restart

Note from the blog post:

You may need to flush the DNS,mDNS and resolver cache, as well as restart your web browsers to clear their internal cache.

After that, ping and nslookup started to agree.

Thanks to harrymc for getting me on the right track.

Aaron Digulla

Posted 2014-01-21T11:14:17.430

Reputation: 6 035

For me it does more sense to configure it right instead of disabling it for local. – keiki – 2014-07-28T08:44:13.997

@otakun85: And how would I do that? – Aaron Digulla – 2014-07-28T08:46:56.823

@AaronDigulla You configured it and the current highest voted answer disabled it for local. – keiki – 2014-07-28T09:16:35.633

This worked for me too, except I could not start the service with domain-name=.alocal, the error message suggested I should remove the dot: domain-name=alocal. This way it worked. – MariusMatutiae – 2014-10-17T15:16:49.787

@MariusMatutiae: What was the error message? – Aaron Digulla – 2014-10-20T08:52:26.023

Neat, that's my blog! Glad it helped you! – spazm – 2015-07-09T01:39:30.517

2This is amazing. Thanks so much. Battled with this for hours before finding this post. – fpghost – 2016-09-29T11:03:30.397

OK and what if there is no avahi on my system... – Petr – 2017-04-19T14:08:44.807

@Petr then either your DNS server has it (and for some reason is configured to use it to resolve names) or someone has broken your DNS or you're wrong and you have avahi. – Aaron Digulla – 2017-05-30T12:53:34.740

1Also note, ping will use nss, nslookup does not. (it uses lwres and, well, bind, to talk to directly with a resolver) – Ricky Beam – 2014-01-28T01:25:33.563

11

Modifying /etc/nsswitch.conf and replacing:

hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4

by:

hosts:          files dns

worked for me.

doep

Posted 2014-01-21T11:14:17.430

Reputation: 211

2

This works at the cost of the AVAHI services

– Aaron Digulla – 2015-04-09T07:42:59.523

8

Easy thing to do: Edit /etc/default/avahi-daemon

Change the line:

AVAHI_DAEMON_DETECT_LOCAL=1

to

AVAHI_DAEMON_DETECT_LOCAL=0

Restart the avahi-daemon, or kill it.

I don't like Avahi, and I don't use any of its features. If you want to truly disable avahi, modify /etc/init/avahi-daemon.conf, similar to the following:

start on (never 
          and filesystem
      and started dbus)
stop on stopping dbus

MikeDawg

Posted 2014-01-21T11:14:17.430

Reputation: 137

2I don't care if you aren't supposed to put "thanks" comments here, I just wanted to express my thanks and how much time and energy you saved me. Thank you so much! I'm using a company VPN to do some emergency work from a mall and could not get some of our sites working despite everything I thought of trying. I wouldn't have been able to figure this out on my own, that's for sure... – eresonance – 2014-11-20T16:43:47.290

7

it seems .local address can't be access in ubuntu.

a solution is to edit /etc/nsswitch.conf and change this line :

hosts:          files mdns4_minimal [NOTFOUND=return] dns

by this :

hosts:          files dns

Dragouf

Posted 2014-01-21T11:14:17.430

Reputation: 171

This sort of worked for me. In my case, the hosts line had "files resolve [!UNAVAIL=return] mdns4_minimal [NOTFOUND=return] dns" and changing it to "files mdns4_minimal [NOTFOUND=return] dns" worked. Thanks for the pointer in the right direction. – Andorbal – 2016-11-24T17:40:33.130

Tried teh other answers above, disabled avahi (which is a relief) and still had the problem. This answer solved it for me. Thanks – Adam Plocher – 2017-01-12T12:01:36.827

3

If you are not doing connection sharing with other devices or VMs thru your computer, you could turn off dnsmasq in Network Manager.

Edit /etc/NetworkManager/NetworkManager.conf and comment the line (put a # in front of it) :

dns=dnsmasq

Then do :

sudo restart network-manager

That will turn off the local resolver.

Source: DNS in Ubuntu 12.04.

harrymc

Posted 2014-01-21T11:14:17.430

Reputation: 306 093

2

So my guess is now that /lib/libnss_mdns4_minimal.so.2 is somehow noticing that the IP address ends with .local and not with .com and then the [NOTFOUND=return] is triggered.

How do I fix this?

Pretty good guess, but the other answers are overkill. The simple solution is to remove the bit that is indeed triggered, i.e. remove just [NOTFOUND=return].

Removing it means that if mdns4_minimal returns NOTFOUND, the next entry on the resolver list is used. This is the normal behavior; [NOTFOUND=return] is an optimization to fail quicker on unknown names but it assumes all .local names are in mDNS.

MSalters

Posted 2014-01-21T11:14:17.430

Reputation: 7 587

1

I had an interesting case with the same symptoms (ping, mount etc. not working, but host, dig working). Check permissions on /etc/resolv.conf file. In my case, someone changed it and I did not have rights to read it (although cat /etc/resolv.conf and editing the file worked fine).

Anyway, strace was showing:

open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)

And as a result, it was trying to query localhost (127.0.0.1) instead of a IP of nameserver from the resolv.conf file:

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
poll([{fd=4, events=POLLOUT}], 1, 0)    = 1 ([{fd=4, revents=POLLOUT}])

And tcpdump was not showing any DNS traffic when pinging. All is working after a permission fix:

# chmod 644 /etc/resolv.conf
# ls -l /etc/resolv.conf
-rw-r--r-- 1 root root 111 Oct  3 09:54 /etc/resolv.conf

Another problem might be extended attributes of the file or any other access problem. In that case just delete the /etc/resolv.conf file and recreate it from the scratch.

Pik Master

Posted 2014-01-21T11:14:17.430

Reputation: 141

Another problem that I had one one server was that all the root folders had lost their x flag, and thus a lot of binaries didn't behave properly. The fix was chmod +x /*. – Silex – 2017-06-12T09:22:31.270

0

Another reason is the format of /etc/hosts. Make sure there are no spaces between IP and host name, instead use a TAB. After changing to TAB the host name could be resolved by ping.

127.0.0.1        test.local
         ^^^^^^^^ → Should be a TAB not multiple spaces.

Thomas Lauria

Posted 2014-01-21T11:14:17.430

Reputation: 119

it could be an answer but it is formatted as comment ... – Francisco Tapia – 2015-07-09T12:44:14.853

-1

Setup avahi-daemon on Ubuntu for so you can reach hostname ubuntu.local from host OS

sudo apt-get install avahi-daemon avahi-discover avahi-utils libnss-mdns mdns-scan

João Pedro Rodrigues

Posted 2014-01-21T11:14:17.430

Reputation: 1

Sorry, this answer is not related to the question. We're not asking how to install Avahi The question is how Avahi can break DNS lookups after it was installed. – Aaron Digulla – 2019-05-07T13:45:11.903