dd , pv , gzip and exclude directory, possible?

Question

Is it possible to do the following:

dd if=/dev/hdx | pv | gzip > /path/to/image.img\
--exclude=/temp --exclude=/lost+found --exclude=/proc --exclude=/path/to/image.img

So I want to make a backup to an image that is compressed with gzip, but I want to exclude some directories and the file to where it is being written, how can I do this?

Answer 1

You should not use dd if you want to exclude some directories and/or files, it's nigh-impossible (without corrupting the filesystem by deleting sectors seemingly at random).

And you can't make a proper dd backup of a mounted partition either, for the same reasons you can't fsck a mounted partition (since it sounds like you're asking "I want to `dd if=sda of=sda")

And you could do this with dd instead of using pv:

Sending a USR1 signal to a running 'dd' process makes it print I/O sta‐ tistics to standard error and then resume copying.

$ dd if=/dev/zero of=/dev/null& pid=$!
$ kill -USR1 $pid; sleep 1; kill $pid

18335302+0 records in 18335302+0 records out 9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s

But, you'd probably be better off using tar with possibly some options like:

 --exclude=PATTERN
       exclude files, given as a PATTERN

 --exclude-tag=FILE
       exclude contents of directories containing FILE, except

 --exclude-tag-all=FILE
       exclude directories containing FILE

 --exclude-tag-under=FILE
       exclude everything under directories containing FILE

 -z, --gzip, --gunzip --ungzip

 -J, --xz

 -T, --files-from FILE
       get names to extract or create from FILE

 -X, --exclude-from FILE
       exclude patterns listed in FILE

See man tar and countless examples on the web.

Unless you're backing up some strange OS or programs that expect files to be in a certain spot on the disk, then you might want a dd copy of a whole unmounted partition/disk, piped to gzip/xz/etc.