How to batch truncate a file by a set amount of bytes?

5

I have ~300 files that need to be truncated (I need to remove a specified number of bytes from the beginning of the file). I can do it one by one using a hex file editor, but given the number of files I have to process that would be a rather overwhelming task.

Is there an automated solution for this? (Operating system is Windows 7 64bit.)

groovy354

Posted 2012-10-08T21:08:41.290

Reputation: 233

1

Your best bet is to use PowerShell. It should be pre-installed with the O.S. I'm not on a Windows machine now, but I'll come back with an answer tomorrow, if noone comes up with one. Here's a clue though: http://www.sans.org/windows-security/2010/02/11/powershell-byte-array-hex-convert

– None – 2012-10-08T21:31:53.460

Answers

5

dd has a skip option.

Per file You can use dd if=MyFile of=my_new_file skip=BytesToSkip

Optionally in a loop (e.g. in bash in the target directory)
for a in * ; do echo processing $a ; dd if=$a of=$a.shorter skip=300 ; done

Adjust skip with the correct number of bytes, KB or MB
If the files are large playing with the block size (bs=X) might speed things up.

Hennes

Posted 2012-10-08T21:08:41.290

Reputation: 60 739

Tell that to Windows... :) – None – 2012-10-08T21:29:55.643

There was no win7 tag yet when I started my answer. Anyway, how big are the files? Small enough for a quick move to a unix host? Large enough to warrant installing extra tools? – Hennes – 2012-10-08T21:44:43.170

6dd is available for Windows too. In CMD the loop could be done like this: for %a in (*.*) do dd if=%a of=%a.shorter skip=300. – Ansgar Wiechers – 2012-10-08T21:46:22.513

2

Here you go...

Powershell code:

$PATH = "d:\My Dir"
$BYTES_TO_TRIM = 10

$files = dir $PATH | where { !$_.PsIsContainer }

foreach ($file in $files) {
    Write-Output "File being truncated: $($file.FullName)"
    Write-Output "  Original Size: $($file.Length) bytes"
    Write-Output "  Truncating $BYTES_TO_TRIM bytes..."
    $byteEncodedContent = [System.IO.File]::ReadAllBytes($file.FullName)
    $truncatedByteEncodedContent = $byteEncodedContent[$BYTES_TO_TRIM..($byteEncodedContent.Length - 1)]
    Set-Content -value $truncatedByteEncodedContent -encoding byte -path "$($file.FullName)"

    Write-Output "  Size after truncation: $((Get-Item $file.FullName).Length) bytes"
    Write-Output "Truncation done!`n"
}

user127350

Posted 2012-10-08T21:08:41.290

Reputation:

This script gives the following error: "Exception of type 'System.OutOfMemoryException' was thrown.." – dtmland – 2014-05-08T19:55:07.087

@dtmland I didn't thought of 100 GB files when I wrote it. If you want you can come with a byte-by-byte reading solution. – None – 2014-05-10T18:26:04.220