13

The behavior of dig +short is to return nothing (null) when running a query that returns no answer (nothing to do with the timeout, just a definite null answer).

It is ok when running a query by itself, but when combining with an -f parameter and running a batch of queries, it is terrible!

dig +short -f queries.txt

queries.txt: A somedomain.com TXT otherdomain.com A somedomain.com

Now, if the DNS server returns nothing for TXT otherdomain.com (i.e., ANSWER: 0), not timeout or something else, then the output of the above dig command will be something like:

dig +short -f queries.txt 1.2.3.4 1.2.3.4

i.e., only two lines. Not suitable for "paste" and other similar commands. You can no longer merge the output of queries.txt and the production output of dig.

Anything elegant can be done here?

xofer
  • 3,052
  • 12
  • 19
Alex
  • 1,768
  • 4
  • 30
  • 51
  • Why are we voting to move this to Unix.SE? Bulk data gathering with dig is very much valid in a professional context. – Andrew B Aug 12 '14 at 13:44
  • I agree with Andrew. dig can be used on any platform, for troubleshooting, data analysis and migrations. Its certainly not nix specific. The first closest candidate answer site is actually SO (scripting) or Networking (is there such site? I did not bother to look, that would be too specialized). – Alex Aug 13 '14 at 00:01
  • by the way, I ended up using a loop for doing this because the output needed to be clean, in a table, with EMPTY (empty answer) result and ERROR result (e.g. timeout) clearly indicated: `sed '1d' dnsrecs.csv | tr , ' ' | while read rec type; do out=$(dig +short $rec $type || echo '(error)' 2>/dev/null) && if [ -z "$out" ]; then echo '(empty)'; else echo $out; fi; done` – Alex Aug 26 '14 at 00:59
  • Be careful with that. `dig` will return error status on `SERVFAIL`, but not `NXDOMAIN`. – Andrew B Aug 26 '14 at 03:17
  • @AndrewB indeed, I've noticed that. However, in my case '(empty)' was ok for both no answer and NXDOMAIN. – Alex Aug 26 '14 at 03:21

3 Answers3

5

There is no real way to make +short do what you want it to in this context. It's simply the wrong tool for the job when working with bulk data.

The solution I found when running into this problem was to use a combination of filters: +noall +question +answer. +noall turns all display fields off, +question displays the query being made with a ; comment prefix, and +answer displays the answer.

The output looks like this:

$ dig +noall +question +answer google.com serverfault.com
;google.com.                    IN      A
google.com.             284     IN      A       74.125.137.101
google.com.             284     IN      A       74.125.137.138
google.com.             284     IN      A       74.125.137.102
google.com.             284     IN      A       74.125.137.100
google.com.             284     IN      A       74.125.137.113
google.com.             284     IN      A       74.125.137.139
;serverfault.com.               IN      A
serverfault.com.        187     IN      A       198.252.206.16

In the event that you get no response back, you will see two adjacent questions. You won't know why the query failed as this output doesn't display a RCODE (neither does +short), but the output is sufficient for analyzing a bulk data set and locating records that need more verbose analysis.

If you find yourself doing bulk analysis of DNS referrals, switch +answer out for +authority.

Andrew B
  • 31,858
  • 12
  • 90
  • 128
3

Something like this is good for pasting as it will preserve exactly one line of output for every line of input in case there is either 1 or 0 responses (which the accepted answer does not). It prints either the IP or the original FQDN. If needed it can be altered to print all the resolved IPs within one line, but I knew in my case there is only one IP.

for i in $(cat list); do
   ip=$(dig +short $i)
   [ -z "$ip" ] && echo $i || echo $ip
done

It saves the answer to a variable and then checks if the variable is empty (not answer, print the input), or not empty (print the output).

Marki555
  • 1,488
  • 1
  • 14
  • 27
0

I think that this python script would work.

from subprocess import Popen, PIPE

def dig(z):
    proc = Popen(
        args='dig +short {} | head -n 1'.format(z),
        shell=True, stdout=PIPE
    )
    return proc.communicate()[0]

result = dict()
with open('queries.txt') as zones:
    for zone in zones.readlines():
        zone = zone.rstrip()
        result[zone] = dig(zone)

with open('queries.txt', 'w+') as results:
    for key, value in result.items():
        if value == '':
            value = 'no answer\n'
        results.write('{} : {}'.format(key, value))
Vasili Syrakis
  • 4,435
  • 3
  • 21
  • 29