How to tell what files are different on two servers

here we can see that md5sums of this two directories are different, how to tell if the folder structures or some certain files are the reason of this difference?

[skynet]~> ssh evn-web04 'find  /www/web/prod/evn.tumo.lab -type f -exec md5sum {} \; |sort -k 34 | md5sum'


Ubuntu 14.04.4 LTS evn-web04.tumo.lab ssh-pty

f247a2e41cf54f14cefe83b8872ae862  -

[skynet]~> ssh evn-web03 'find  /www/web/prod/evn.tumo.lab -type f -exec md5sum {} \; |sort -k 34 | md5sum'
Ubuntu 14.04.4 LTS evn-web03.tumo.lab ssh-pty

53413a49fb754210666f4292e8b9ee14  -

Edik Mkoyan

Posted 2016-03-17T12:16:38.363

Reputation: 327

Answers

Generate the list of files on one server, and check it on the other. Repeat with the servers in reverse order, and collate the lists.

First, gather a list of files and their corresponding hashes on evn-web04:

ssh evn-web04 'find  /www/web/prod/evn.tumo.lab -type f -exec md5sum {} \;'

Second, pass this to the same hash utility on evn-web03, passing --check, and filter for anything that is not identical:

ssh evn-web04 'find  /www/web/prod/evn.tumo.lab -type f -exec md5sum {} \;' | \
ssh evn-web03 "md5sum --check - | grep -vE ': OK$'"

This will give you a list of files that:

exist on evn-web04, but do not exist on evn-web03, or
exist on evn-web04, and are different between evn-web04 and evn-web03

To get a full list of differences, you need to also reverse the server order (because files may exist on evn-web03 that do not exist on evn-web04, which the above won't catch as-is). Just run the same commands again but put switch places of the server names.

a CVn

Posted 2016-03-17T12:16:38.363

Reputation: 26 553

How can I generate the list of files in given sub directory, I mean how can I get the full tree and also probably exclude some files or folders. – Edik Mkoyan – 2016-03-17T13:25:14.037

@EdikMkoyan Just tweak the parameters to find to suit your needs. md5sum --check will look strictly at the list of files it is fed (which is also why you need to do this twice to make sure both servers are fully in sync). – a CVn – 2016-03-17T13:27:36.490

The following one-liner will do:

diff <(find /path/to/dir -type f -exec su -c 'echo -n {}; md5sum {}' \;) \\
 <(ssh me@remote find /path/to/dir -type f -exec su -c 'echo -n {}; md5sum {}' \;)

It is often under-appreciated that this kind of tasks can be carried out with a single command, thanks to process substitution.

MariusMatutiae

Posted 2016-03-17T12:16:38.363

Reputation: 41 321

This is a good suggestion. I do this type of things every once in a while myself; don't know why I didn't think of it as a possible solution to the OP's problem. – a CVn – 2016-03-17T13:28:43.717

My first thought was to use

rsync ... --dry-run path user@remote:/path

From the man page

-n, --dry-run               perform a trial run with no changes made

Rsync would report any files that are different

RedGrittyBrick

Posted 2016-03-17T12:16:38.363

Reputation: 70 632

But I also nee to check if the content of two directories are the same – Edik Mkoyan – 2016-03-18T09:00:44.237