Script to reference another file first

Question

Hi I have a script and in it I need to get the hostnames from IPs that the script generates. The script will generate those IPs into 1 neat file with only 1 column. It is literally just IP address (1 for each line). I have another file called 'db' that looks like this:

IP Hostname

'db' is my database that stores a list of IP address and hostnames that are delegated to those IPs. These are static address and will not change, usually servers or switches, etc. What I need is for the IP addresses that the script generates, I'd like to resolve them with the hostnames FIRST from this database file and if it doesn't find it there, then run the host command. I am already doing the latter with host but I really want them coming from my db file first. The final output only needs to be 1 file with 1 column of hostnames. I have some kind of ideas using awk with arrays but can't seem to quite get exactly what I need.

Answer 1

grep is the command whose primary purpose is to look through text for a pattern (or patterns).  Its simplist application to this task would be

grep 1.2.3.4 db

which will output the “IP_address Hostname” line(s) from the db file that match the pattern “1.2.3.4”.  Note that I said line(s) (rather than line) and match the pattern (rather than contain the string), because there are a couple of gotchas here.

• One is that grep, as I've said, (by default) searches for patterns rather than strings.  These patterns are called regular expressions, and there's a world of documentation on them.  The most important fact about regexs for this discussion is that a period, “.”, is a pattern that matches any single character.  So, for example, the pattern (regex) 1.2.3.4 would match the strings 132.364 and 1a2b3c4.  These are probably not an issue as far as the IP_address column is concerned, but such strings might appear in the Hostname column.
• Another is that, unless otherwise specified, grep searches for the provided pattern anywhere on the line.  So the pattern 1.2.3.4 would match lines containing 31.2.3.4 and 1.2.3.42.

But I believe that you’ll do a good job of getting the line you want – and only that line – with the command

grep "^1.2.3.4[[:space:]]" db

where

• “^” is another character that’s special in regular expressions; it means that this pattern must appear at the beginning of a line.

• “[[:space:]]” represents a whitespace character; either a space or a tab.  If you are sure that your db file uses only spaces, you can just search for a space; e.g.,

  grep "^1.2.3.4 " db
  

These will still find lines that begin with 132.364 or 1a2b3c4.  But, if your db file is properly structured, you should have no such strings at the beginnings of lines in your db file.

You can prevent “.” from matching arbitrary characters with any of the following:

• grep "^1\.2\.3\.4[[:space:]]" db
• grep "^1\.2\.3\.4 " db
• grep -F "1.2.3.4 " db

but the first two will require you to jump through hoops in your script to convert 1.2.3.4 to 1\.2\.3\.4, and the third one takes away the special meaning of “^” (so 31.2.3.4 will still match).

As stated above, these commands all match the entire line from the db file that matches the IP address.  To get just the hostname (the second column), use

grep "^1.2.3.4[[:space:]]" db | awk '{print $2}'

This will just write the hostname to the standard output (typically, the screen).  A more useful operation in a script would be

db_host=$(grep "^1.2.3.4[[:space:]]" db | awk '{print $2}')

which captures that output into the shell variable db_host.

So, finally, this section of your script might look something like

db_host=$(grep "^$this_IP_address[[:space:]]" db | awk '{print $2}')
if [ "$db_host" = "" ]
then
    Use the host command.
          ︙
fi

which you would put in a loop, wherein the variable this_IP_address takes on each IP address that you want to process.  If you have them in a file (i.e., another file, separate from db), then read that file one line at a time.

For example, if you have IP addresses (one per line) in a file called src_ip, and all you want to do is write the hostnames to the standard output, you could say,

while read this_IP_address
do
    db_host=$(grep "^$this_IP_address[[:space:]]" db | awk '{print $2}')
    if [ "$db_host" = "" ]
    then
        # Use the host command to lookup $this_IP_address.
        host "$this_IP_address" | head -n 1 | awk '{print $5}'
    else
        echo "$db_host"
    fi
done < src_ip

grep -f src_ip db wouldn’t be particularly useful for the problem you’re describing (if I understand it correctly) because it would report all the lines in db that match any IP address in src_ip – leaving you to figure out which IP addresses weren’t matched.

---

A simpler way of doing this has occurred to me:

awk -v this_one="$this_IP_address" '$1 == this_one {print $2}' db

which copies shell variable this_IP_address into awk variable this_one and then says “print the second field of any line whose first field is equal to this_one.”  This is one process, as opposed to two, and it eliminates the issue of “.” matching arbitrary characters.  Wrap it in db_host=$(…) as before.