2

I am researching a remote off-site backup strategy. The major limitation is that the upstream pipe is only ~50 kb/s, so with that in mind the incremental backup capabilities using file differentials of rdiff-backup seemed like the proper tool.

In a test case:

  1. First rdiffbackup was run on a local directory to a different local directory. In the directory was a 10GB pax archive of a Maildir directory.

  2. The 10GB pax archive was replaced with a 12GB pax archive in the source directory (representing a month's worth of additional emails). rdiff-backup was run again. I expected rdiff-backup to be faster this time because there was only a 2GB difference in file size, however, not only was the runtime longer but in the target directory there were 2 files, the original 10GB pax archive and a 12GB temp file.

Can rdiff-backup incrementally backup compressed archive files? It doesn't seem to be able to.

Currently the mail server writes pax Maildir backups to an external drive. But instead of using these as the rdiff-backup source, would it be better to let rdiff-backup just backup /home/%user%/Maildir (many, many small files)?

I suppose that if the external drive were to fail it would be better not to cripple the 2nd backup system as well!

cheers,

kilolima

edit : couldn't add a tag for 'pax'?!

kilolima
  • 121
  • 4

1 Answers1

3

You're better off to not compress the data.

rdiff-backup does analyze files for differences, but if they're compressed archives, it may not be able to find any differences and thus be forced to store the entire new file again.

Also, you can use ssh -C to compress the ssh connection and save some bandwidth.

Finally, if possible, you should get some more bandwidth; that's barely better than dialup (or maybe it is dialup?). Backing up 12GB of data would take weeks over dialup, and even the 2GB difference could take days.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • It's 50 KILObytes per second. Dial-up is about 5 kb/s. There is no way to increase upload speeds without paying hundreds of more $$ a month. And only 30 miles away from Silicon Valley! – kilolima Aug 16 '12 at 21:12
  • OK, in that case the bandwidth isn't TOO bad. – Michael Hampton Aug 16 '12 at 21:18
  • No, not too bad but it makes the use of a cloud backup service impossible! :( But thanks for the heads-up on rdiff and not using archives. Do you think if I run a rdiff-backup of /home to an external drive, and then move and connect that external drive to the remote NAS and copy the rdiff directory to the NAS's drive, the next time rdiff-backup is run from the server over SSH to the NAS it will be fooled into not trying to copy over everything again? – kilolima Aug 16 '12 at 21:52
  • Only if you leave the backup on the external drive. – Michael Hampton Aug 16 '12 at 21:53