7

I am deploying my first tape backup solution (with Quantum Superloader 3 LTO-7, CentOS 6.9), and I am surprised and stymied by the what seems like daunting configuration requirements of both Bacula and Amanda.

I am attracted by the philosophical approach that Amanda takes regarding using existing common tools and formats. This is how I've been doing my disk-based backups for years. The problem is that when I look for hints online, all I can find are posts from the early 2000s (precious few around 2010) about backing up only a few GB to tapes of only a few dozen GB. Further, the parameters as they exist on a fresh yum install amanda-server of the configuration files in /etc/amanda/DailySet1/amanda.conf seem aligned with these types of numbers (e.g. bumpsize is 20MB!). This seems totally out of touch with modern reality. I am dealing with multiple highly compressible 40+ TB file systems and 6 TB tapes on a 16-tape autoloader.

Of course, the anachronism wouldn't matter by itself, but I am at a loss regarding how to configure parameters (e.g. dumpcycle, runspercycle, tapecycle, bumpsize, etimeout) in the absence of any collective wisdom on the web. The defaults seem abysmal. Also, as a matter of course, rather than rotating many small backups across many large tapes, I have a mix of many small backups that fit on a tape necessarily coexisting with large backups that will have to span several tapes, and I need to be sure tapes aren't being overwritten. The tapes in the autoloader definitely have sufficient capacity for multiple level 0s and several level 1s, but I am not sure how to configure it.

So my questions:

  1. Is Amanda used for jobs of this magnitude? Is it superseded at this point by Bacula? Is there a compelling reason to use one of the other?
  2. It almost seems simpler and less risky to just write my own tape-changing logic and tar/dump/dd/rsync invocation script than to rely on massive configuration files with dozens of parameters? Foolhardy or advisable long-term? Do people do this?
  3. I want to be able to rotate a subset of tapes out of the autoloader for offsite backup. Will this totally flummox Amanda or Bacula or cause administration headaches?
  4. Can these systems take advantage of the bar codes I have on the tapes given that the hardware reads the bar codes?

I apologize if these questions seem uninformed. I've been reading literature for days now, but this is opaque to me. Any help or guides getting started with dozens of TB backups on modern autoloaders with modern tapes on a modern OS would be very much appreciated!

rg6
  • 185
  • 1
  • 11

4 Answers4

3

I use Bacula since about 13 years, and Quantum Superloader LT6 Tape Library since about 3-4 years.

  1. Is Amanda used for jobs of this magnitude? Is it superseded at this point by Bacula? Is there a compelling reason to use one of the other?

Our backup volume is one 25TB storage, and a few TB other smaller folders. Changes are not big, so incrementals are fast. The biggest problem I have is that one full job with 25TB would take a few days to complete, blocking the other jobs. That means you need to split the job to more manageable junk sizes. LTO-7 is faster but your amount of data is also bigger.

We had Bacula working before and integration of the tape library was not that difficult so we stick with it. I am quite happy with the stability and features, restores are not difficult once you get it. No experience with Amanda, though.

  1. It almost seems simpler and less risky to just write my own tape-changing logic and tar/dump/dd/rsync invocation script than to rely on massive configuration files with dozens of parameters? Foolhardy or advisable long-term? Do people do this?

Sure you can do it, but Amanda and Bacula have the kinks worked out mostly. Some write a list of files and their checksums in the first blocks, the tar archive after that. Tar can do multi-volume archives out-of-the-box (checked with GNU tar 1.29).

  1. I want to be able to rotate a subset of tapes out of the autoloader for offsite backup. Will this totally flummox Amanda or Bacula or cause administration headaches?

Yes that certainly works, Bacula has an "update slots" command to update which tapes are in the library. It will tell you at backup time which tape you need to load in case it is missing. We do that for a small backup job and need to change a tape every month out of a pool of 7 tapes. The other 6 are always offsite (offsite and more space in the library). For a restore we bring what we need, but luckily we seldom need a restore.

  1. Can these systems take advantage of the bar codes I have on the tapes given that the hardware reads the bar codes?

Yes, using "label barcodes storage=LTO6" will use the barcodes to label the tapes. There are websites around to print your own labels.

What I never tried is LTFS, i.e. a file system on tape, maybe combined with rsnapshot could be a feasible alternative.

I strongly suggest to try out all the solutions, and do not be afraid to look at old documentation, a lot is still valid. And do restores!

mrossi
  • 538
  • 4
  • 8
3

I've used Amanda for many years now, Bacula more recently. They'll both get the job done. They both have somewhat of a learning curve. Both are very reliable.

I'd say overall, Bacula is a bit easier to configure. Bacula uses MTX for all its tape interface so you can have it do just about anything you want. I use it with Amazon VTL primarily.

Amanda can definitely handle your workload, and I'd argue is a bit more flexible once you've got the hang of it.

kervin
  • 211
  • 2
  • 7
0

Amanda 3.5 Binaries are available in the following pages.

Source tarballs are available from

http://www.amanda.org https://sourceforge.net/project/showfiles.php?group_id=120

Binaries for many systems are available from

http://www.zmanda.com/download-amanda.php

Documentation can be found at

http://wiki.zmanda.com

Here's a list of the changes for release 3.5 (from the NEWS file): Look at the ReleaseNotes and ChangeLog file for more details.

Fix compilation on Solaris Do not check all 'r' bit on suid binary Fix parsing of configuration override (-o) can unset some setting client code will not fail if shared memory is not available amreport lot of improvement allow '*' for a datestamp wildcard amgetconf print an empty string if a parameter is not set instead of 'no such parameter' amdump new --no-dump, --no-flush and --no-vault argument amstatus fix lock holding disk to protect multiple parallel access

Mithun raj
  • 7
  • 2
-1

I ended up here researching open source backup solutions for a project. I've come to pretty much the same conclusion, they seem to leave a lot to be desired and have many caveats you might not run into before using them extensively or trying to actually restore. Configuration seems to be quite complex and confusing.

I don't know what your project is, but a suitable commercial solution would save you a lot of time and headache. I recommend Archiware P5, it's simple, robust, reliable and not that expensive compared to big enterprise vendors. Compared to the time saved on configuration you might actually save money, unless you count your time spent as free. I believe with P5 you would be done in a few hours and that includes getting to know the software and testing how it works. The actual backup configuration is very simple to do in a web interface.

One thing they don't have is plugins for different databases and hypervisors that the big brands have, but sounds like you wouldn't need those anyway. In a way lack of plugins is a good thing because it simplifies the overall backup process and there are other tools to handle those special needs. P5 allows you to run pre/post backup scripts to dump db's, shutdown services or what have you.

You can get a free trial from their site without having to talk to a sales rep, they get that trying out their software is the best sales pitch. I don't work for Archiware, I'm just a happy user.

nixer
  • 1