11

We have one big problem at the moment: We need to mirror a filesystem for one of our customers. Thats usual not really a problem, but here it is:

On this filesystem there is one folder with millions of hardlinks (yes! MILLIONS!). rsync requires more then 4 days to just build the filelist.

We use the following rsync options:

rsync -Havz --progress serverA:/data/cms /data/

Has anyone a idea how to speed up this rsync, or use alternatives? We could not use dd as the target disk is smaller then the source.

UPDATE: As the original filesystem is ext3 we will try dump and restore. I will keep you up2date

Thomas Berger
  • 1,700
  • 12
  • 22

3 Answers3

3

We have used ext* dump now. Works well, and the restore side doesn't even have to be ext*.

We have done an offline backup, by umounting the device and used dump vf - /dev/vg0/opt | gzip -c > /mnt/backup/ext3dump.gz.

Here the last lines you could see size, time, speed and the last inode numbers:

DUMP: dumping regular inode 47169535
DUMP: dumping regular inode 47169536
DUMP: Volume 1 completed at: Wed Jun 29 05:42:57 2011
DUMP: Volume 1 54393520 blocks (53118.67MB)
DUMP: Volume 1 took 4:16:43
DUMP: Volume 1 transfer rate: 3531 kB/s
DUMP: 54393520 blocks (53118.67MB)
DUMP: finished in 15403 seconds, throughput 3531 kBytes/sec
DUMP: Date of this level  dump: Wed Jun 29 01:24:29 2011
DUMP: Date this dump completed:  Wed Jun 29 05:42:57 2011
DUMP: Average transfer rate: 3531 kB/s
DUMP: DUMP IS DONE
Tobu
  • 4,367
  • 1
  • 23
  • 31
Thomas Berger
  • 1,700
  • 12
  • 22
  • I don't know if this is still true but dump used to have some issues if the file-system is in use at time of dump. Since your goal is speed I suppose you have already disabled all other access to it, but just in-case.. Let us know how you go – SuperBOB Jun 28 '11 at 09:46
3

You need to upgrade both sides to rsync 3. From the changelog:

- A new incremental-recursion algorithm is now used when rsync is talking
  to another 3.x version.  This starts the transfer going more quickly
  (before all the files have been found), and requires much less memory.
  See the --recursive option in the manpage for some restrictions.

It has been over 2 years since rsync 3.0.0 was released, but, unfortunately, most enterprise distributions are based off code older than that, which means you're probably using rsync 2.6.

For reference (if anyone else is having this problem), if you are running rsync 3 already, then you are using options that are incompatible with incremental recursion. From the man page:

    Some options require rsync to know the full file list, so  these
    options  disable the incremental recursion mode.  These include:
    --delete-before,   --delete-after,    --prune-empty-dirs,    and
    --delay-updates.

Also, again, both sides must be running rsync 3 for incremental recursion to be supported.

Steven Pritchard
  • 776
  • 5
  • 3
  • Pritchard thank you for that hind, but the incremental parts are no problem, both sides uses rsync > 3.0. If we use rsync without -H we have a great speed improvement, but thats not what we need. – Thomas Berger Jul 01 '11 at 15:18
  • Ouch. Yeah, in that case you might want to look into options to speed up filesystem access (like switching to ext4 if you're using ext3), switching to faster disks or RAID levels (if that's even an option), etc. Unfortunately, you might be at a point where the filesystem just can't be fast enough and block-level backups may be your only option. I had this problem trying to rsync a BackupPC pool from one server to another. – Steven Pritchard Jul 01 '11 at 17:10
0

You could use LVM and take snapshots of the volume, then rsync the snapshot as the backup.

Alternatively, you could combine this with the other answer and use dump on the snapshot volume, to avoid having to take the original volume offline.

Teddy
  • 5,134
  • 1
  • 22
  • 27
  • Anything that works at a block level, not file-system level would probably be a huge improvement. – Marcin Jun 28 '11 at 14:59
  • As you could see in my question, i have to mirror across the Network, not local. Also LVM is NOT a mirror, its just, as you said, a snapshot. – Thomas Berger Jun 28 '11 at 16:19
  • 1
    @Thomas Berger: My thought was that you would then copy the snapshot (using rsync) over the network. And how exactly do you define *mirror*, if an LVM snapshot is not one? – Teddy Jun 29 '11 at 13:08
  • That still has the same problem: It would take days. In this days there would be a huge dalta (not that we would need that) so we have to reserver enough space, and we don't have that space. And a mirror is a independent copy of source. We have to copy the data from production to development for the customer. – Thomas Berger Jun 29 '11 at 13:24
  • @Thomas Berger: I originally meant that you would rsync the actual snapshot volume, *not* the file system on the snapshot. However, I now believe the snapshot+dump solution to be better. – Teddy Jun 29 '11 at 13:59
  • Have an upvote for bringing snapshots into the picture. This is an improvement no matter what else is done. – user239558 Sep 22 '14 at 12:14