How can I diff two config files?

Question

I've got two snmpd.conf files, one on a server that works, and one that doesn't. How can I diff the two config files while stripping out irrelevant comments and newlines?

Answer 1

If you are somewhat comfortable with vim, I would strongly encourage you to use vimdiff:

vimdiff file1 file2

This will open a vim session with two panes, with one file on each side. Highlights and color will indicate differences between the files, and all identical parts will be hidden (folded, but expandable).

Then, if you want to selectively merge differences from one file to the other, you can use the following commands:

(Consider the "current file" to be the one where the cursor is)

^W^W to change focus from one file's window to the other file's window

]c to advance to the next block with differences

[c to reverse search for the previous block with differences

do (diff obtain) to bring changes from the other file to the current file

dp (diff put) to send changes from the current file to the other file

Note: Both do and dp work if you are on a block or just one line under a block.

u to undo

zo to unfold/un-hide text

zc to re-fold/re-hide text

zr will unfold both files completely (use :help fold for more about folding )

:diffupdate will re-scan the files for changes

As you start moving changed text over or bringing changes in, the now-identical parts of the files will automatically fold, too.

When you are finished, you can quit and write both files with :xa!

You can also write, quit, discard changes, etc., one pane at a time just as you would normally do with vim.

You can use all the common vim commands to edit the files at will ; I've only described the most common and useful commands you're likely to use in a vimdiff session (as opposed to a generic vim one).

Answer 2

diff <(grep -v '^#' f1) <(grep -v '^#' f2)

To avoid blank lines, and lines containing nothing but spaces, in addition to identical lines that have a single difference of added leading spaces...

diff -b \
  <(grep -vE '^([ \t]*#|^[ \t]*$)' f1)\
  <(grep -vE '^([ \t]*#|^[ \t]*$)' f2)

By this point though, I'd probably put that into a script and write something like the original suggestion that's a little more readable.

Answer 3

Beyond Compare is the ultimate tool for this!

Link: http://www.scootersoftware.com/

Available for Windows and Linux.

Jeff wrote a good overview article about the tool awhile back:
http://www.codinghorror.com/blog/archives/000454.html

Answer 4

Expanding on nima's one-liner, you could do that as a shell function and drop it in your .bashrc

diff <(grep -v '^#' f1) <(grep -v '^#' f2)

becomes (using -u because I like unified diffs)

function cleandiff {
  diff -u <(grep -v '^#' $1| grep -v '^ *$') <(grep -v '^#' $2 | grep -v '^ *$')
}

If you like GUI diff viewers, meld is nice, and understands revision controlled dirs/files.

Answer 5

After cleaning the comments, I would advise using KDiff3, it's a pretty good diff/merge tool and you dont need vim fu to use it :)

Answer 6

There might be a more elegant way to do it, but pragmatically (and quickly):

grep -v '^#' server1-snmpd.conf | grep -v '^ *$' > server1-snmpd.conf-clean
grep -v '^#' server2-snmpd.conf | grep -v '^ *$' > server2-snmpd.conf-clean
diff server1-snmpd.conf-clean server2-snmpd.conf-clean

Answer 7

If you're using a bash-like shell, you can try this:

# Name this diff-stripped
STRIPPED=
for i in $*; do
    egrep -v "^#|^\s*" "$i" > "$i.stripped"
    STRIPPED="$STRIPPED $i.stripped"
done

diff $STRIPPED

Then invoke it like this:

 diff-stripped file1 file2 ...

You can also change diff to vimdiff or gvimdiff which both come with vim.

Answer 8

Extending Xerxes' solution, you can use more sophisticated tools than diff for displaying the differences.

wdiff

wdiff can be "too smart" at times, but I find it often useful for taking a quick glance at differences between configuration files. This script can be used for output with colors:

#!/bin/bash

RED=$'\e'"[1;31m"
GREEN=$'\e'"[1;32m"
RESET=$'\e'"[0m"
WDIFF_ARGS="-w$RED -x$RESET -y$GREEN -z$RESET --avoid-wraps"

wdiff $WDIFF_ARGS \
  <(grep -vE '^([ \t]*#|^[ \t]*$)' $1) \
  <(grep -vE '^([ \t]*#|^[ \t]*$)' $2) \
  | less -R

On Ubuntu and other Debian-based systems, just apt-get install wdiff before using this script.

Meld

Meld is a nice GUI alternative, but its "Text filtering" feature has some issues. Instead of using text filtering, I remove comments altogether before showing the results in Meld. The drawback is losing the ability to edit the files while comparing them. Here's a simple script for using Meld:

#!/bin/bash

meld <(grep -vE '^([ \t]*#|^[ \t]*$)' $1) \
     <(grep -vE '^([ \t]*#|^[ \t]*$)' $2)

Answer 9

Sometimes, several additional common lines can be stripped of by sorting files prior to the diff, so I would add to what already written the following:

 diff <(grep -v '^#' f1 | sort) <(grep -v '^#' f2 | sort)

this of course makes sense for files where lines' order does not effects it's content (so be aware).

Answer 10

This is the same as nima's one liner, but will filter out blank lines too as somebody requested.

diff -u <(egrep -v '^(#| *$)' f1) <(egrep -v '^(#| *$)' f2)

(I'd also install colordiff if possible and use that in place of normal diff)

Answer 11

I use WinMerge http://winmerge.org to diff files, granted I have to pull them down to my machine, but it works for.