Is there any way to speed up ddrescue?

Question

I had a 500GB drive HDD crash about 5 days ago. I used ddrescue on the important partition a few days ago, and it's been on "Trimming failed blocks" for almost 2 days now.

Original command:

ddrescue -n /dev/rdisk1s2 /Volumes/OSXBackup/rdisk1s2.img /Volumes/OSXBackup/rdisk1s2.log

Current output:

Initial status (read from logfile)
rescued:   248992 MB,  errsize:   1007 MB,  errors:   15867
Current status
rescued:   249021 MB,  errsize:    978 MB,  current rate:    17408 B/s
   ipos:    44405 MB,   errors:   15866,    average rate:     2784 B/s
   opos:    44405 MB,     time from last successful read:       0 s
Trimming failed blocks...

The original command used the ddrescue -n parameter, and I have restarted the process a few times as needed (and it seemed to pick up right where it left off each time).

Is there any way to speed up this process?

Edit: Six hours later, this is the current status:

rescued:   249079 MB,  errsize:    920 MB,  current rate:      409 B/s
   ipos:    39908 MB,   errors:   15851,    average rate:     2698 B/s
   opos:    39908 MB,     time from last successful read:       0 s
Trimming failed blocks...

It appears that while "errors" is counting down excruciatingly slowly, ipos/opos is counting down how much data it has to churn through, and it seems to be working at a rate of 750MB/hour. At this rate, it will complete in ~53 hours. Yikes.

Edit #2: Two days later, still running. However, there is hope. It has moved passed the "Trimming failed blocks" portion, and on to the next phase "Splitting failed blocks". If anything, what should be taken away from viewing this question is that this definitely takes a long time when a good amount of data/errors are involved. My only hope is that I can successfully recover some important data when all is said and done.

rescued:   249311 MB,  errsize:    688 MB,  current rate:        0 B/s
ipos:    26727 MB,   errors:   15905,    average rate:     1331 B/s
opos:    26727 MB,     time from last successful read:      20 s
Splitting failed blocks...

Answer 1

I observed that using the -n (no-split) option together with -r 1 (retry once) and setting -c (cluster size) to a smaller value can help.

My impression is that the splitting step is very slow as ddrescue splits and splits again the damaged areas. This takes a lot of time because ddrescue tries to restore very small portions of data. So, I prefer to use -n (no-split) together with -c 64, -c 32, -c 16, a.s.o.

Probably the -n (no-split) should always be used for one first pass in forward and reverse directions. It seems that the more the data were split, the slower the cloning, although I'm not sure about this. I assume the larger the non-treated areas, the best when running ddrescue again, because more contiguous sectors are to clone.

As I'm using a logfile, I don't hesitate to cancel the command with Ctrl+C when the data read speed becomes two low.

I also use the -R (Reverse) mode and after a first pass it often gives me higher speeds reading backwards than forward.

It's not clear to me how already retried sectors (-r N) are handled when running the ddrescue command again, especially when alternating forward (default) and reverse (-R) cloning commands. I'm not sure if the number of times they were tried is stored in the logfile and probably the work is done again useless.

Probably the -i (input position) flag can help speed up things too.

Answer 2

It can be very hard to see the progress of ddrescue, but there is another command included called ddrescuelog.

A simple command ddrescuelog -t YourLog.txt will output these nice infos:

current pos:     2016 GB,  current status: trimming
domain size:     3000 GB,  in    1 area(s)
rescued:     2998 GB,  in 12802 area(s)  ( 99.91%)
non-tried:         0 B,  in    0 area(s)  (  0%)

errsize:     2452 MB,  errors:   12801  (  0.08%)
non-trimmed:   178896 kB,  in 3395 area(s)  (  0.00%)
non-split:     2262 MB,  in 9803 area(s)  (  0.07%)
bad-sector:    10451 kB,  in 19613 area(s)  (  0.00%)

You can even use it while ddrescue is running...

Answer 3

I have found that playing with the -K parameter you can speed things up. From what I've seen if ddrescue finds an error when running with the -n option tries to jump a fixed amount of sectors. If it still can't read it jumps double the size. If you have large damaged areas you can indicate a big K value (for example 100M) and so the jumping on an error will be larger the first time and it will be easier to avoid problematic areas quickly in the first past.

By the way, there is a wonderful graphical application to analyze the log.

http://sourceforge.net/projects/ddrescueview/

Answer 4

One more way to monitor ddrescue's progress (on Linux, at least) is through the use of strace.

First, find the PID for the ddrescue process using "ps aux | grep ddrescue"

root@mojo:~# ps aux | grep ddrescue
root     12083  0.2  0.0  15764  3248 pts/1    D+   17:15   0:04 ddrescue --direct -d -r0 /dev/sdb1 test.img test.logfile
root     12637  0.0  0.0  13588   940 pts/4    S+   17:46   0:00 grep --color=auto ddrescue

Then run "strace" against that process. You'll see something like:

root@mojo:~# strace -p 12083
Process 12083 attached - interrupt to quit
lseek(4, 1702220261888, SEEK_SET)       = 1702220261888
write(4, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
lseek(3, 1702220261376, SEEK_SET)       = 1702220261376
read(3, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
lseek(4, 1702220261376, SEEK_SET)       = 1702220261376
write(4, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
^C

...and so on. The output is fast and ugly, so I then pipe it through "grep" to filter out the stuff I care about:

root@mojo:/media/u02/salvage# nice strace -p 12083 2>&1|grep lseek
lseek(4, 1702212679168, SEEK_SET)       = 1702212679168
lseek(3, 1702212678656, SEEK_SET)       = 1702212678656
lseek(4, 1702212678656, SEEK_SET)       = 1702212678656
lseek(3, 1702212678144, SEEK_SET)       = 1702212678144
lseek(4, 1702212678144, SEEK_SET)       = 1702212678144
lseek(3, 1702212677632, SEEK_SET)       = 1702212677632
lseek(4, 1702212677632, SEEK_SET)       = 1702212677632
lseek(3, 1702212677120, SEEK_SET)       = 1702212677120
lseek(4, 1702212677120, SEEK_SET)       = 1702212677120
lseek(3, 1702212676608, SEEK_SET)       = 1702212676608
^C

In that example, the "1702212676608" equates to "the amount of data that still needs to be processed on that 2 Tb disk you're trying to salvage." (Yeah. Ouch.) ddrescue is spitting out a similar number -- albeit as "1720 GB" -- in its screen output.

strace gives you a MUCH higher granularity data stream for you to examine; it's one more way to evaluate the speed of ddrescue and estimate a completion date.

Running it constantly is probably a bad plan since it would compete with ddrescue for CPU time. I've taken to piping it to "head" so I can grab the first 10 values:

root@mojo:~# strace -p 4073 2>&1 | grep lseek | head

Hope this helps someone.

Answer 5

If your aim is to obtain the bulk of the data intact, then you could speed up its extraction. But if you really want to rescue as much data as possible, then letting ddrecue nibble at each and every is the route to take.

Answer 6

What is the file system of the hard disk where you save the rescue image and the logfile? I just made the experience that rescuing a 500GB internal hard drive (connected via SATA) on a Laptop running Linux Mint from a USB Stick, saving the rescue image and logfile on an exFat formatted USB hard drive, was starting rather slowly (1-2MB/sec) but after around 250GB it was only crawling at <100KB/sec. It seemed to become slower the larger the rescue image file was growing.

Then I moved the rescue image and logfile to another temporary place, re-formatted the USB hard drive with the ext4 file system, moved the files back on it and resumed the ddrescue process - and now it runs with 1-20MB/sec again (fluctuating but around 7MB/sec on average)!

Seems like exFat does not play very well with very large files (several hundred gigabytes).

Answer 7

For a faster and quick option to rescue the disc you can use a sh script file and run the file with "sh filename.sh". It contains this line shown, just repeat "sudo ddrescue" and "sleep 3" few more times, the sleep is used to make the drive rest some seconds, it can be good for some reasons:

#! /bin/sh -e  
sudo ddrescue -d -r0 -e +0 -T 1s -n /dev/drivepartition file.img log.logfile 
sleep 3

The -r0 is with no replies. The -e +0 is for exit on 1 error. The -T 1s exits with 1 second fail read. There are options that can be used as -d for direct and -n for no scrape which can speed up.

You can use -R after finish with option -A once, that will reverse and remove all errorsize and start again backwards. Means it will read errors differently.

Answer 8

dd_rhelp is a shell script that uses dd_rescue "[...] on your entire disc, BUT it will try to gather the maximum valid data before trying for ages on bunches of badsectors"

it's quite old (2012) but still works. haven't tried ddrescue yet.