4

The data is currently stored in a 9TB RAID array, with many users having many files and directories. We'd like to back it up to five 2TB external hard drives, but since we only have one free USB port, this must be done sequentially. This means that once hard drive #2 is plugged in, #1 is unplugged, and we won't know what was originally on hard drive #1 unless the software remembers it.

One solution would be to assign every file or directory to a hard drive beforehand, and then follow through with this as they are connected. Is there a straightforward way to do this?

Nicolas Kaiser
  • 165
  • 1
  • 4
  • 16
Chris
  • 143
  • 3
  • Do you have any special requirements such as *if one drive dies, the data on the rest must be recoverable* or *there is only a small staging area for the data before it is written to the external drives*? – Ladadadada May 15 '13 at 19:38
  • Why does only one USB port dictate that you have to make the copy sequentially? Wouldn't a USB hub solve that problem? The answer to your actual question is below (I'm going with sysadmin1138's answer). Of course, depending on your data, you could just manually break it up into five groups and copy each one to a drive (e.g. copy user accounts a-e to drive one, f-k to drive two, and so on). And, depending on the data, if you back it up with compression it might actually all fit on one or two drives. – Jed Daniels May 15 '13 at 23:36

3 Answers3

6

Most commercial backup-software packages have support for a "removable media" option, which is what you're talking about here. It'll do a backup, prompt for operator assistance (swap drives) and continue the backup. These packages maintain a state database to keep track of which data is stored on what media, so you don't have to.

If you're in UNIX land, ye olde tar has media management options. Spanning big-big data across multiple tapes is kind of what it was built for.

If you're using XFS on Unix, xfsdump has media-management in it as well, above and beyond what tar can do.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
3

http://wiki.bacula.org/doku.php?id=removable_disk

This document describes how to utilize removable disk drives as backup media for a backup solution using Bacula. Having seen many people ask about the use of USB disk drives as a backup media on the Bacula User's e-mail list, I felt that producing this document based on my experience with using USB disk drives as backup media might be beneficial to some bacula users.

What follows is the description of a method to use several USB disk drives as the backup media for a backup solution using Bacula and is based on my experience with setting up a backup system for a small company with three servers and around 10 workstations. This method involves emulating a traditional magazine-based tape library device by using partitions on the USB disk drives as the “magazines”. As such, the method would work equally well for disk drives in Firewire, eSATA, or any other hot-swappable enclosures supported by the OS.

I think this may be an example of what you are looking for. As stated before, any backup suite will have this option.

If you're limited to rsync then I'm guessing that manually preparing lists of files for each drive would be an option.

Community
  • 1
Kw4s
  • 31
  • 2
1

This is pretty standard and any backup solution should be able to handle it, no different then switching out tapes, you are simply backing up to a spanned volume.

Matt Bear
  • 874
  • 3
  • 12
  • 28