unbound/nsd returning SERVFAIL resolving local LAN DNS. nsd alone works fine

Question

I've been using unbound as a local recursive DNS server. Just added nsd to set up local LAN DNS. nsd is listening on port 53530 and that works fine:

$ dig @127.0.0.1 data2.datanet.home -p 53530

; <<>> DiG 9.9.2-P2 <<>> @127.0.0.1 data2.datanet.home -p 53530
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59577
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;data2.datanet.home.            IN      A

;; ANSWER SECTION:
data2.datanet.home.     600     IN      A       192.168.1.62

;; AUTHORITY SECTION:
datanet.home.           600     IN      NS      ns1.datanet.home.

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53530(127.0.0.1)
;; WHEN: Mon Jun 15 07:16:24 2015
;; MSG SIZE  rcvd: 81

When going through the local unbound it does not work:

$ dig @127.0.0.1 data2.datanet.home

; <<>> DiG 9.9.2-P2 <<>> @127.0.0.1 data2.datanet.home
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 47645
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;data2.datanet.home.            IN      A

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jun 15 07:18:02 2015
;; MSG SIZE  rcvd: 47

Here is what I'm getting in the unbound log with verbosity: 4

Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: validator operate: query router.datanet.home. A IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: validator: pass to next module
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: mesh_run: validator module exit state is module_wait_module
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: iterator[module 1] operate: extstate:module_state_initial event:
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: process_request: new external request event
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: iter_handle processing q with state INIT REQUEST STATE
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: resolving router.datanet.home. A IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: request has dependency depth of 0
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: use stub datanet.home. NS IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: cache delegation returns delegpt
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: DelegationPoint<datanet.home.>: 0 names (0 missing), 1 addrs (0 r
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug:    ip4 127.0.0.1 port 53530 (len 16)
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: iter_handle processing q with state INIT REQUEST STATE (stage 2)
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: resolving (init part 2):  router.datanet.home. A IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: use stub datanet.home. NS IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: iter_handle processing q with state INIT REQUEST STATE (stage 3)
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: resolving (init part 3):  router.datanet.home. A IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: iter_handle processing q with state QUERY TARGETS STATE
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: processQueryTargets: router.datanet.home. A IN
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: processQueryTargets: targetqueries 0, currentqueries 0 sentcount
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] info: DelegationPoint<datanet.home.>: 0 names (0 missing), 1 addrs (0 r
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug:    ip4 127.0.0.1 port 53530 (len 16)
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: attempt to get extra 3 targets
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: skip addr on the donotquery list ip4 127.0.0.1 port 53530 (len 1
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: No more query targets, attempting last resort
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: configured stub servers failed -- returning SERVFAIL
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: store error response in message cache
Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: return error response SERVFAIL

In particular what's with this? [1947:0] debug: skip addr on the donotquery list ip4 127.0.0.1 port 53530 (len 1 That would seem to be key, but I'm really not sure why it's saying that.

Here is my entire unbound.conf:

server:
  interface: 127.0.0.1
  interface: 192.168.1.50
  use-syslog: yes
  username: "unbound"
  directory: "/etc/unbound"
  trust-anchor-file: trusted-key.key
  access-control: 192.168.1.0/24 allow
  verbosity: 2
  local-zone: "1.168.192.in-addr.arpa" nodefault
remote-control:
  control-enable: yes
  control-interface: 127.0.0.1
  control-port: 8953
  server-key-file: "/etc/unbound/unbound_server.key"
  server-cert-file: "/etc/unbound/unbound_server.pem"
  control-key-file: "/etc/unbound/unbound_control.key"
  control-cert-file: "/etc/unbound/unbound_control.pem"

stub-zone:
  name: "datanet.home"
  stub-addr: 127.0.0.1@53530
#  stub-first: yes
stub-zone:
  name: "1.168.192.in-addr.arpa"
  stub-addr: 127.0.0.1@53530

nsd.conf has a lot of comments so not sure if I should paste it but in any case nsd seems to work fine. It's pretty much the same as the included example conf except changing the port, enabling the control stuff, and adding the zones.

I'm stumped by this, so any ideas would be appreciated!

For questions on Unbound and NSD, you can use the mailing lists run by NLnet Labs (https://www.nlnetlabs.nl/support/mailing-lists/). On these mailing lists, there is a great community that is very responsive (including the software developers of Unbound and NSD). — Benno Overeinder, Jul 01 '19 at 10:18

score 16 · Accepted Answer · edited May 22 '19 at 07:37

16

This line from the log indicates the problem:

Jun 15 06:12:39 pizza.yoderdev.com unbound[1947]: [1947:0] debug: skip addr on the donotquery list ip4 127.0.0.1 port 53530 (len 1

Unbound by default refuses to send any DNS queries to localhost. To enable it to query localhost, set the do-not-query-localhost to no in the server-section of the Unbound configuration:

server:
  interface: 127.0.0.1
  interface: 192.168.1.50
  [...]
  do-not-query-localhost: no

See the documentation for unbound.conf for a description of the option.

edited May 22 '19 at 07:37

Jan Zerebecki

138
6

answered Jun 15 '15 at 14:25

olav

366
2
4

well that looks right but my quick testing is not showing that as solving it. I'm at work now though and will look more when I get home. Thanks! – Micah Yoder Jun 15 '15 at 15:21
This is odd indeed: With that change, the above dig to unbound (the second one) returns the exact same result to the user (SERVFAIL). But when I look at the unbound log generated at the instant of that query, it shows that it got the correct answer from the upstream. There is no indication in the log of a SERVFAIL response! – Micah Yoder Jun 16 '15 at 00:44
Unfortunately no suggestions from me. All I can say is that removing the `do-not-query-localhost` option on my nameserver gives me SERVFAIL when looking up a stub zone, while inserting it again makes it work again. – olav Jun 16 '15 at 06:26
guess I'll accept this and dig some more and perhaps ask a follow-up on what is still going on .... – Micah Yoder Jun 16 '15 at 11:49
unbound's cache even shows the nodes but they aren't being returned to the resolver! :/ – Micah Yoder Jun 16 '15 at 11:54
2

+1 to answer: Upvoting because this answer gave some clear advice that helped. Maybe my situation wasn't identical to the original posting, but the advice still helped lead to a quick resolution on something that wasn't working. – TOOGAM Oct 15 '15 at 11:34
This ended the hours of troubleshooting over 2 days! Unbound has too many options! I thought I had a firewall misconfiguration among other things. I finally gave up and searched Google for "nsd unbound SERVFAIL" and found this solution! Who knew moving NSD to localhost ONLY would be such a pain in the ass. – Clint Pachl Jan 26 '22 at 06:19
The unbound error I was getting: "error: SERVFAIL : all the configured stub or forward servers failed, at zone pachl.us. no server to query nameserver addresses not usable have no nameserver names". Hopefully this helps other Googlers. – Clint Pachl Jan 26 '22 at 06:23

Markus Ueberall · Answer 2 · 2020-11-07T15:57:00.537

2

I ran into the same problem in split-horizon DNS context–the Unbound log indicated that an "incoming scrubbed packet" (obtained from NSD) contained the IP address/CNAME entry in question, but after "finishing processing", the latter would not be passed through.

Eventually, adding the equivalent of domain-insecure: "datanet.home" solved this for me using Unbound v1.12.0 and NSD v4.3.3.

edited Nov 07 '20 at 15:57

answered Nov 07 '20 at 15:49

Markus Ueberall

121
7

score 0 · Answer 3 · answered Sep 25 '22 at 02:46

I had a similar error messages with an almost identical configuration, except that I had the following option:

    tls-upstream: yes

That option caused unbound to expect upstream queries for both forwarded and stub zones to be over TLS only for transport. However, my NSD authoritative server hosting the stub zones were configured for local connections only and without TLS. This may also cause the SERVFAIL response.

The proper configuration was to set tls-upstream to no and instead set the forward-tls-upstream to yes within the forward-zone section if the stub zones are not also configured for TLS transport.

unbound/nsd returning SERVFAIL resolving local LAN DNS. nsd alone works fine

3 Answers3