7

There have been several questions related to providing backups, but most of them were either too specific, intended for home usage or Windows-based.

What I'd like to hear:

  • How do you ensure that all of your Linux servers are properly backed up?
  • How often do you backup your servers?
  • What do you backup, apart from the obvious /home directories?
  • How do you ensure that backups are incremental, but still easy to restore?

Especially the last question has been troubling me. Compressing backups into tarballs works okay, but causes a serious performance issue when I suddenly need to restore a backup. On the other hand, when not compressing the backups and syncing them to another server, there is a serious risk of creating inconsistent user permissions.

What tools help you to make this as easy as possible and what is your preferred approach to the matter?

Aron Rotteveel
  • 8,239
  • 17
  • 51
  • 64

7 Answers7

4

How often do you backup your servers?

The standard backup is done nightly, there are a few backups that are done more frequently.

How do you ensure that all of your Linux servers are properly backed up?

The backup send a report with a good subject on success or failure. I check that I have received the status messages at least a couple times a week.

What do you backup, apart from the obvious /home directories?

Depends on the server, but I almost always want /etc, /root, and /var/lib/dpkg. I then add any data directories.

How do you ensure that backups are incremental, but still easy to restore?

I use Dirvish,

Dirvish is a fast, disk based, rotating network backup system.

With dirvish you can maintain a set of complete images of your filesystems with unattended creation and expiration. A dirvish backup vault is like a time machine for your data.

Dirvish uses rsync and the --link-dest option to hard-link identical files for each backup together. The backup is fast, it doesn't waste space, and restoring is as simple as copying files.

Zoredache
  • 128,755
  • 40
  • 271
  • 413
  • That's neat. I'd never heard of Dirvish. Looks rather like TimeMachine on OS X. – Stephen Darlington May 01 '09 at 12:24
  • @Stephen Darlington, the difference is that OSX TimeMachine is based on filesystem events instead of a scheduled backup. Dirvish/rsync where available long before rsync, and many think it is what inspired Apple. – Zoredache May 01 '09 at 18:04
2
* How do you ensure that all of your Linux servers are properly backed up?

I run duplicity on them, backing up to amazon S3.

* How often do you backup your servers?

I do a full backup quarterly and nightly incrementals. This is mostly to cut down on storage costs at S3.

* What do you backup, apart from the obvious /home directories?

/home, /etc and /var. I run a debian box so I also do a nightly 'dpkg --get-selections >/etc/debian.pkgs' so I can track what's installed.

* How do you ensure that backups are incremental, but still easy to restore?

duplicity does that well. I test it occasionally by doing a restore from some point in history.

pjz
  • 10,497
  • 1
  • 31
  • 40
1
  • They get backed up completely automatically, and all backups are monitored to ensure that they completed successfully (as reported by the backup software). This monitoring is crucial, as it "closes the loop" on backups to ensure that they're running successfully.
  • Usually daily, although some systems (database servers) do occasionally get more interesting strategies.
  • Package lists, /etc, /var and /home. Everything else can be recreated from there.
  • I use dirvish at home, which just creates hardlinked trees of files with rsync. It does a great job, but does require rsync running as root (to ensure permissions are maintained) and is hard on the filesystem from on inode consumption perspective. At work we use rdiff-backup, and while it has it's own set of ideosyncracies, stores permissions in metadata files, so you don't need root permissions to write the backup (but do need to run an explicit restore to get the permissions back).
womble
  • 95,029
  • 29
  • 173
  • 228
1

I have finally set up a RAID6 server together with rsnapshot which is using rsync and hardlinks between different backups, so this saves place but makes restore a simple copy operation.

I run incremental every night, rotating the last 14 snaphots, accumulate nightly into to a set of every second weeks snapshots, and then quaterly sets and yearly sets. I back up everything on my disks.

For offline backup I have just bought a SAS LTO4 tape streamer which I just have received but not installed yet, but I plan then store some of the rsnapshot backups from time to time and store the tapes at my office.

hlovdal
  • 1,075
  • 11
  • 18
1

How do you ensure that all of your Linux servers are properly backed up?

We run rdiff-backup nightly. We have a script that lives in /usr/local/bin that cron runs every night.

To ensure it has worked, we use nagios to check that the copy of /var/log/messages is never older than 48 hours. If it is we can see that on the nagios webpage and in the emails nagios will send to administrators.

How often do you backup your servers?

Every night.

What do you backup, apart from the obvious /home directories?

We backup everything, apart from a few bits that are excluded - /tmp /proc /sys and some things in /var/log/

How do you ensure that backups are incremental, but still easy to restore?

We use rdiff-backup (also see this tutorial). rdiff-backup produces a directory with the full up-to-date copy of whatever you are copying, but keeps the diffs with previous backups hanging around, so you can recover files for as far back as the diffs go. You will probably want to delete the old diffs occasionally (to save space) and there is an rdiff-backup command for this which could be run weekly by cron.

Hamish Downer
  • 9,142
  • 6
  • 36
  • 49
1

I have a cron job running on the servers that run rdiff-backup.

/etc/ gets backed up, /home/ and any other directory that will ensure me getting another one up and running asap when it goes down.

Everything is gathered in one central place and all of this goes to tape every night. Tape gets collected the next morning and goes away (off site) for 2 weeks before it's put back in rotation.

For the very crucial things, I have a live spare running. As in: a second server that sits there and idles (and is my toy basicly) but has all the files/services of the first one. When the first one goes down, this one can take over in a flash, all that needs to be done is change a DNS record.

Gert M
  • 1,471
  • 1
  • 15
  • 14
1

I've been exploring this myself and as I want automatic, offsite backups to a storage service, Amazon Glacier and Backblaze B2 seem like good options right now (Backblaze being the cheaper one currently).

Looking at the clients that support these services and the features you'd expect (encryption, deduplication, incremental backups), there seem to be two options that work on Linux:

  • Duplicity - open source
  • HashBackup - you need a paid annual license for HachBackup in addition to your storage service
metakermit
  • 143
  • 1
  • 7