How to tell which parts of a binary file are changing

1

Is there a tool I can see to see which parts of the file are being modified? I've seen tools that show me which segments of my hard drive are being accessed, but I want something similar for a single file, if something like that exists.

I'm encrypting a file and I want to see if, once a part is written to disk, that part ever changes again. For example, once the first MB of the file is encrypted and saved to disk and the program moves on, is anything in that first MB updated once encryption is done.

As some detailed context, I have a process where I use gpg4win to encrypt a very large file (36GB), and then I have to FTP it to a remote server. Both steps of this process take many hours, but the FTP takes a bit longer. If encryption just keeps extending the file, but the beginning remains unchanged, then I can start the FTP process as soon as the file is just a little ways in. If the header on my encrypted file gets updated at the end, though, I have to wait until that steps is complete before I start the upload.

SqlRyan

Posted 2014-07-01T15:02:22.963

Reputation: 1 021

There hundreds of hex editors out there. It might be easier to encrypt a similar but smaller file instead of try to load two 36GB files into memory. – Ramhound – 2014-07-01T15:49:31.857

@Ramhound I'd test with a smaller file, so I suppose I'm asking for concept - just wanted to explain my end goal. I've rearranged my question a bit so it's more clear. – SqlRyan – 2014-07-01T16:02:43.617

Possible duplicate of Binary Diff Tools On Windows or How do I compare binary files in Linux? – Breakthrough – 2014-07-01T16:08:42.757

Answers

2

Usually that is some variant of a rsync like algorithm: 

* split a file into blocks of x bytes
* build a hash over these x bytes
* compare the hash for each block between both files
* transfer the changed blocks

This obviously works only if the file does not change all the blocks when it is written. Thus, this cited paragraph is something to keep in mind:

Disk encryption often uses special purpose modes specifically designed for the application. Tweakable narrow-block encryption modes (LRW, XEX, and XTS) and wide-block encryption modes (CMC and EME) are designed to securely encrypt sectors of a disk. (See disk encryption theory)

OpenGPG is a mixed crypto system (it mixes asymmetric and symmetric ciphers to achieve it's goal), right now I find nothing about the used ciphers AND their application for block level deltas. You might want to pick a different crypto solution (eg, openssl), based upon these observations: http://www.daemonology.net/blog/2009-06-11-cryptographic-right-answers.html

That said:

are both tools to calculate a binary diff and merge the parts together later on.

Related: Incremental (delta) backup of a encrypted data

akira

Posted 2014-07-01T15:02:22.963

Reputation: 52 754

Asked: 2014-07-01T15:02:22.963

Viewed: 224 times

Active: 2014-07-01T16:04:12.610