How to cut a file to a given size under Linux?

Question

I want to shrink a file's size by brute-force, that is, I don't care about the rest, I just want to cut the file, say by half, and discard the rest.

The first thing that comes to mind is Perl's truncate. I'm following the example on that page and did the exactly the same thing:

seq 9 > test.txt
ls -l test.txt
perl -we 'open( FILE, "< ./test.txt" ) && truncate( FILE, 8 ) && close(FILE);'

But the file still has the same size:

$ ls -lgG test.txt
-rw-rw---- 1 18 2013-08-08 09:49 test.txt

How can I make this work?

Answer 1

You may want to use the truncate command:

truncate --size=1G test.txt

SIZE can be specified as bytes, KB, K, MB, M, etc. I assume you can calculate the desired size by hand; if not, you could probably use the stat command to get information about the file's current size.

Answer 2

perl -we 'open( FILE, "< ./test.txt" ) && truncate( FILE, 8 ) && close(FILE);'

opens the file for reading. However, to truncate the file you need to modify it, so a read-only file handle isn't going to work. You need to use the "modify" mode ("+>").

As a side issue, it always amazes me when people let system calls fail silently and then ask what went wrong. An essential part of diagnosing a problem is looking at the error message produced; even if you don't understand it, it makes life much easier for those you ask for help.

The following would have been somewhat more helpful:

perl -we 'open(FILE, "<", "./test.txt") or die "open: $!";
          truncate(FILE, 8) or die "truncate: $!";
          close(FILE);'

although admittedly that would only have reported "invalid argument". Still, that is useful information and might well have led you to the conclusion that the open mode was wrong (as it did for me).

Answer 3

You can use tail to cut last 100000 bytes, example:

tail -c 100000 file > file2

the -c outputs the final 100000 bytes of the file, for more options:

man tail

To replace original file with the file you just generated:

mv file2 file

Answer 4

The answer above citing truncate is nice. dd will also do the job:

dd if=test.txt of=test2.txt bs=1 count=8
mv test2.txt test.txt

Answer 5

there is a completely different way to do this, with bash, using the ed program. the following script will retain only the last 5000 lines of all files sitting in the specified directory. this can easily be modified to loop over several directories, change the number of lines, etc.

#!/bin/bash

LOGDIR=/opt/log
SAVELINES=5000

dirs="$LOGDIR"
for dir in $dirs ; do
    files=${dir}/*
    for f in $files ; do
        echo -e "1,-${SAVELINES}d\nwq" | ed $f 1>/dev/null 2>&1
    done
done