What does dd do that pv doesn't?

4

2

I see examples online all the time of using dd in combination with pv to show progress... But, something I don't understand is that pv is perfectly capable of doing the task entirely on it's own. So, why use dd as well? Is there some advantage dd provides that pv doesn't?

Examples using both dd and pv. Some of these are really complicated... 

dd if=/dev/sda | pv | dd of=/mnt/backup.img
dd if=/dev/sda | pv -s 10G | dd of=/mnt/backup.img
dd if=/dev/sda | (pv -s `fdisk -l /dev/sda | grep -o '[0-9]*\{1\} MB' | awk '{print $1}'`m) | dd of=/mnt/backup.img

You can acheive the same thing using pv alone by doing

pv /dev/sda > /mnt/backup.img

Drew Chapin

Posted 2015-06-11T15:30:35.970

Reputation: 4 839

"Some of these are really complicated" - Seems like a good enough reason to just use pv. I searched on the topic, most results, seem to indicate there isn't a built in way to report the progress. So either dd has added this feature since those statements were made. But pv also seems to communicate between two processes.

– Ramhound – 2015-06-11T15:50:05.297

can't find a soure, but I'm pretty sure pv stands for Pipe-View, since it just seems to show statistical data on piping/redirecting operations. In fact, I'm certian that the pv command listed would work in exactly the same way without the pv invocation, other than that you wouldn't get the nice progress output. – Frank Thomas – 2015-06-11T15:51:34.657

I didn't like the title @barlop made since I know what pv is doing, I just don't understand what dd is doing that pv isn't. So, I changed the title to "what does dd do that pv doesn't?" – Drew Chapin – 2015-06-11T16:08:52.780

@Ramhound, but in my example, pv /dev/sda > /mnt/backup.img, pv is not reading from a pipe? It's reading from the device file. – Drew Chapin – 2015-06-11T16:10:01.320

@druciferre - Your other three examples do pipe the stdout. Its been awhile, and I forget what function > performs but I do know the results in communication between two processes. – Ramhound – 2015-06-11T16:14:44.297

ockquote>

is output redirection. same as piping but to a file instead of a subsequent process. the PV does nothing more than cat the file, and count bytes and give you a progress bar. its the > in that command thats actually moving data.

– Frank Thomas – 2015-06-11T16:16:37.247

2

@druciferre: You should take a look at this related interesting question: Is dd better than cat?

– Karan – 2015-06-11T16:32:31.720

@FrankThomas - Redirect, i knew it was something, had to write my own limited kernel with pipe and redirect support was never able to get both working :$ – Ramhound – 2015-06-11T16:41:13.817

@FrankThomas pv is indeed 'pipe viewer' – Sirex – 2015-06-11T19:59:32.103

@Karan +1 that is a fantastic link. I would note that many have pointed out a difference between cat and dd being that dd you can configure block size. Though pv can also. One of the answers there shows a way to monitor block size with the ltrace command. You can specify block size with e.g. pv -s 2M – barlop – 2015-06-11T23:04:13.890

yes your new title "What does dd do that pv doesn't?" is much better than any prior one. – barlop – 2015-06-11T23:16:23.293

This is a brilliant question and the current answer to it seems to be incorrect. It deserves a better, more comprehensive answer. – Hashim – 2017-10-18T23:48:56.550

Answers

4

Well, pv just provides sight into a pipe operation, so the real question is, is the pipe/redirection equivalent to dd. the answer of course is no.

While simple cases of dd usage may appear to work similarly to IO redirection or piping, dd is a rather complicated tool with a lot of features related to file data conversion (piping barely knows what a file is, let alone what kind of byte order/encoding it is) whereas redirection just pushes bits, and pv just watches data going through a pipe, with no more understanding of it than just counting bytes.

dd understands disk structures like blocks and addresses, which pv has no knowledge of. dd interfaces with the kernel at the driver level, while pv occurs above a great deal of IO abstraction.

for instance, dd could grab the backup of the MBR of a disk (always in the 63rd block) with:

dd if=/dev/sda of=MBR_boot.img bs=512 count=1 skip=62

There is no clean way to do that with redirection/piping.

Frank Thomas

Posted 2015-06-11T15:30:35.970

Reputation: 29 039

What happens if pv runs into an error while reading? I know what happens when you use dd. It stops... Also, it should be pv does have a flag to specify how many bytes to read. so you could backup the first 512 bytes, but you wouldn't be able to skip the first 62. – Drew Chapin – 2015-06-11T22:28:50.340

1I just did man pv, and it has stuff about blocks – barlop – 2015-06-11T22:43:59.683

@druciferre what i'm going to say doesn't answer what you said but may be of interest to you. ddrescue is very good it reports progress nicely – barlop – 2015-06-11T22:48:48.067

1check karan's link showing an answer on differences between dd and cat, Its main point is you can't specify block size in cat. Thing is, you can in pv. One answer suggests the ltrace command to monitor block size e.g. you can see dd famously defaulting to 512 bytes. Doing this in ubuntu $ ltrace pv </dev/zero >/dev/null shows pv using 128KB block size. You can say pv -s 2M and then it uses 2MB So pv totally lets you specify block size and it's in man pv. – barlop – 2015-06-11T23:07:29.730

Asked: 2015-06-11T15:30:35.970

Viewed: 3 574 times

Active: 2015-06-12T10:50:53.177