Find sequence in files using HEX Editor

2

A few days ago, I decided to defragment and optimize my 2TB external Hard Disk using Auslogics Defragmenter. Now, a little background here. The partition that I defragmented was an 1.8TB partition encrypted by VeraCrypt, a fork of TrueCrypt. After the defragmentation was complete, I found that several files have been corrupted. The files in which I could detect the corruption were all compressed files, compressed using Windows 10's default compression. I opened the files up in WinHex, and I saw a curious thing: The end of all these corrupted files had a block of "DF DF DF" sequence. Here is the Screenshot of the problem. Please note that, though the picture shows it's an arc file, it was still compressed by Windows compression. The curious thing is that, even though the files had those DF chunks, a Binary Search of the mounted partition did not yield any such sequence.

Now I want to create a batch file that will:

  • Create a list of compressed files on the disk
  • Check for DF sequence at the end of binary content of each file
  • Make a list of all affected files

I tried to solve this using Powershell script but was unsuccessful. I think approaching the problem in Linux platform would be much easier, but I have no idea how to do it. Any help or suggestion would be greatly appreciated.

Additional Note: Many have said that the Binary Search of the mounted partition was unsuccessful because of the fact that the partition is encrypted. That's wrong. I mounted the partition before the search. Hence, the search was not on RAW data of the HDD but the data of the unencrypted partition.

Additional Details: OS: Windows 10 Professional X64 / Ubuntu 14.04 x64 File System: NTFS

Sabyasachi Mukherjee

Posted 2016-07-29T19:45:51.633

Reputation: 55

I can give you only guess. In Hex DF is the char 233, in some codepage (set) the letter Ú. It said few. If you see it in binary, the pattern DF =11011111 seems to be one of the patterns used to wipe securely the HDD. If something get wrong and that segment was overwritten for security reasons... Try nonetheless to understand how it works the defragment tool you used on an encrypted partition. Good Luck. – Hastur – 2016-07-29T20:57:59.127

What I think is that PerfectDisk was already running on the background and, when I ran a second defragmenter, somehow there was a conflict between the two defragmenters, and the compressed files got the brunt of it. – Sabyasachi Mukherjee – 2016-07-29T22:04:08.390

Answers

2

The Get-Content cmdlet with the -Encoding Byte and -Tail switches gives you what you want to match against in PowerShell. Assuming this is where it fails you.

kRush

Posted 2016-07-29T19:45:51.633

Reputation: 276

Here is a link which deterred me from using the -Encoding Byte switch. Can this tool do the same task?

– Sabyasachi Mukherjee – 2016-07-29T21:59:52.543

Would it help if I told you the 250k rep Microsoft MVP is talking out his ass? The tool looks like it could do it but unless you actually try something and provide specifics where it fails you I'm outta here, especially seeing how this is basically a dupe of your own older question. – kRush – 2016-07-30T00:03:50.667

Well, I mounted the partition in read-only mode and tried the Get-Content cmdlet, and it worked! Regarding the matter of this being a dupe question, the previous question was regarding the recovery of data, and this is merely regarding the search for files which are damaged. – Sabyasachi Mukherjee – 2016-07-30T09:45:26.997

@SabyasachiMukherjee I suppose that now you can even upvote the answer... :-) ... Not all the people want to be an unsung hero :-)

– Hastur – 2016-07-30T23:29:15.040

I don't have the requisite reputation points to cast a public upvote. Sorry. – Sabyasachi Mukherjee – 2016-07-30T23:41:56.113

0

Get-ChildItem -recurse -file -Attributes Compressed -path $Path | Foreach-Object -Process { if ((Get-Content -Tail 1 -encoding String -literalpath $_.fullname).endswith("ßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßß")) { Write-Output "$($_.Name) `t $($_.FullName) `t $($_.length)"}  }

Simple one liner. Basically based on what kRush said. Replace $Path with the correct path.

Mycroft Holmes

Posted 2016-07-29T19:45:51.633

Reputation: 35