Intentionally reduce speed of ddrescue

1

0

(first time poster) I want to slow down how fast ddrescue copies data. I'm imaging a hard drive from my linux laptop to my windows desktop. ddrescue is able to run faster than my computer can download what it gives, so its choking out my computer's internet. I image things like this occasionally, and want to know if there's a way to choke ddrescue down a little, either from the Windows desktop, or the linux laptop. (Windows 10, Parted Magic)

Thanks for your help?

McKittrick Swindle

Posted 2015-05-28T01:43:42.847

Reputation: 11

probably answer to this How to Throttle per process I/O to a max limit? is what you may use.

– MolbOrg – 2016-05-11T23:58:12.930

Answers

0

I have no simple solution for ddrescue.
EDIT: I have not-so-simple solution now.

Preparing target file

You will need to know the source hard drive size in 512-bytes unit. Store it in a variable.

SIZE=$(sudo blockdev --getsz /dev/sdX)

The target file should have appropriate size beforehand.

fallocate -l $((512*$SIZE)) /mnt/samba/share/target.dd

Preparing delayed device

First you need a loop device.

sudo losetup -f /mnt/samba/share/target.dd
sudo losetup -a

The second command tells you which loop device is associated with target.dd. Here I assume it is /dev/loop0.

Then you create a device with device mapper.

echo "0 $SIZE delay /dev/loop0 0 0 /dev/loop0 0 500" | sudo dmsetup create delayed_target

The number 500 in this example states that every write operation on /dev/mapper/delayed_target will be delayed by 500 ms. (See man dmsetup and this document for details.)

Actual reading

sudo ddrescue --force -D -c 2048 /dev/sdX /dev/mapper/delayed_target logfile.log

The switch -c 2048 tells ddrescue to process 2048*512 bytes at a time, that's one megabyte. Because of -D program omits caching etc. which would gather many write operation into single one. With -D every megabyte forms separate write operation which hits 500 ms delay.

Tweaking

One tweak is to change -c 2048 in ddrescue invocation. Another – to change the delay in dmsetup (see Cleaning section).

You can always stop ddrescue with Ctrl+C, tweak the delay if needed, rerun ddrescue with the same logfile and maybe another -c parameter – and it will continue. Experiment to find values that fit your needs.

Cleaning

sudo dmsetup remove delayed_target

(To tweak the delay go back to dmsetup create now.)

sudo losetup -d /dev/loop0

Note

Instead of delaying writes to target file you can delay reads from source device. It seems to be a better way since there is no need for loop device. However, my tests indicated that there are immediate reads from newly created mapper device. These reads hit the delay many times and block dmsetup itself. My interpretation may be all wrong, still there is significant delay after dmsetup create. It makes tweaking (if not all the procedure) virtually impossible.


My original answer:

From my experience ddrescue is significantly better than dd only if there is a read error. If your disks are generally in good shape and you expect read errors to be rare, you may use dd with pv.

Example:

dd if=/dev/sdb conv=sync,noerror bs=32M 2>dd.log | pv -L 4M > /mnt/samba/desktop/my-image.dd

The noerror parameter for dd makes it continue after read error occurs, if any; sync causes dd to output zeros in that case, so the good data afterwards is placed with proper offset within the image file. I prefer large bs for hdd performance. Note that with such a large bs one read error may cause as much as 32MiB of erroneous zeros in the image, so you may need to:

  • either manually play with pocket calculator and dd's bs, skip, seek and conv=notrunc options in order to do what ddrescue does automatically: read and save as much as possible, hopefully reducing erroneous zeros to 512B or 4096B per read error (these numbers are common sector sizes for disks).
  • or manually play with pocket calculator and create a logfile for ddrescue, then run ddrescue to reread only problematic fragments of the source; refer to info ddrescue on logfile structure.

In both cases a saved stderr from dd will be useful to locate sectors with problems, hence 2>dd.log redirection.

Good thing with pv is its ability to change speed limit on the fly (replace PID with actual pid of the original pv used with dd):

pv -L 6M -R PID

This way you can adjust the limit without starting over.

Kamil Maciorowski

Posted 2015-05-28T01:43:42.847

Reputation: 38 429