Poor rsync performance over ESATA

Question

I'm trying to back up some of my stuff to an ESATA based drive.

I'm seeing really slow transfer speeds however:

rsync -avzP ...
someFile
     1062838 100%    1.96MB/s    0:00:00 (xfer#1371, to-check=1099/118283)
someFile
     1062751 100%    1.42MB/s    0:00:00 (xfer#1372, to-check=1098/118283)
someFile
     1027193 100%    1.08MB/s    0:00:00 (xfer#1373, to-check=1097/118283)
someFile
     1093821 100%  967.56kB/s    0:00:01 (xfer#1374, to-check=1096/118283)
someFile
     1103236 100%    3.48MB/s    0:00:00 (xfer#1375, to-check=1095/118283)
someFile
     1204130 100%    2.23MB/s    0:00:00 (xfer#1376, to-check=1094/118283)
someFile
     1187734 100%    1.53MB/s    0:00:00 (xfer#1377, to-check=1093/118283)
someFile
     1115959 100%    1.15MB/s    0:00:00 (xfer#1378, to-check=1092/118283)
someFile
     1137257 100%  992.50kB/s    0:00:01 (xfer#1379, to-check=1091/118283)
someFile
       32768   3%  264.46kB/s    0:00:03

But hdparm tells me this drive could do oh-so-much more:

jazz home # hdparm -Tt /dev/sdd

/dev/sdd:
 Timing cached reads:   984 MB in  2.00 seconds = 491.45 MB/sec
 Timing buffered disk reads:  280 MB in  3.01 seconds =  93.07 MB/sec

One wrinkle that could be causing this is that I've got LVM set up and I'm copying from a RAID5 vgroup. But should I really be such crushingly slow performance out of this?

This is on an ext4 partition on an Ubuntu box, across to another ext4 partition on the ESATA drive.

Is there anything I can do to speed this up?

Answer 1

Are you sure it's actually being slow? Most file transfer programs give unreliable results for transfers that take less than a few seconds. What's your aggregate throughput?

If it really is being slow, tt's highly unlikely this is an eSATA problem. LVM has very low overhead, and RAID 5 shouldn't hurt you at all for reads. Make sure the filesystem is mounted with relatime (or noatime), as that's a common source of thrashing for rsyncs.

Answer 2

So, it looks like the killer in my case was the 'z' parameter for the rsync command. Switching away from using z for compression, shot me up to the 20-30 MB/second mark.

Good to know - I'm guessing I should only be using compression when my bandwidth is the limiting factor.

Answer 3

mount -o sync (or noasync) really hurts performance, but is sometimes used on removable media to reduce the damage that may occur on an unexpected unplug. I don't use

Whether your disk is mounted synchronously by default depends on your hardware, your software's ability to recognize that the hardware is eSATA as opposed to internal SATA, and your configuration. I'm not really sure what Ubuntu would do. But if that's the case (and it's easy to check by reading the output of mount), you can just unmount and remount without sync.