9

I am looking for a solution for backing up about 30 of my servers that are a mix of centos, windows 2003 and 2008 machines.

I have a san in another data center with a 1 Gig link between the sites that im going to be backing up onto so this is purely a disk based backup solution.

What I am ideally looking for is some open source software possible with a nice gui that will allow me to have a backup cycle with full backups on a monthly basis that go back 6 months and weekly backups that are saved for one month. I'm hoping for de duplication as well so if a file is saved on the first backup the software creates a symlink to it rather then saving the data again in all the proceeding backups. It must also be smart enough to know when a file is deleted that the file is moved to the latest backup it exists in and all symlinks to it are updated.

I was looking at bacula but im unsure if it can do the deduplication or something similar. If it can will someone be kind enough to point me at a decent guide as to how to do it.

If it does not can someone recommend another solution open source or not?

user71173
  • 133
  • 1
  • 1
  • 3
  • I think bacula has deduplication, but it might only be available with the enterprise version of it. It was causing us some issues so our team decided to turn it off to fulfill a security requirement. – user1207381 Dec 03 '15 at 22:41

6 Answers6

9

Something simple like rsnapshot may do what you want. It doesn't sound like you want what's traditionally referred to as deduplication. The synthetic full backup functionality using hard-links that rsnapshot provides sounds like exactly what you're asking for.

(On the traditional deduplication front you may want to look at the Opendedup project as a starting point. It's the most complete open source data deduplication project that I've seen. It uses a custom filesystem (SDFS) as the basis of its functionality. I tend to think that this isn't want you're looking for, but in the interests of completeness I mention it here.)

Evan Anderson
  • 141,071
  • 19
  • 191
  • 328
  • Rsnapshot does seem like the sort of software im after. Is there any decent gui tools where i can setup scheduled snaphots from a centralised location? I'm thinking of having monthly backups that i retain for 1 year and weeklies that i retain for a month. – user71173 Feb 19 '11 at 15:44
3

Bacula writing to a ZFS filesystem that has de-duplication turned on, then write the snapshot from that host to a remote host with ZFS and apply that transaction log to that filesystem.

user9517
  • 114,104
  • 20
  • 206
  • 289
chris
  • 11,784
  • 6
  • 41
  • 51
  • 2
    sounds massively over complicated. From what i'm reading about backup the most important thing is K.I.S.S – user71173 Feb 19 '11 at 15:50
  • @user71137 -- the question was how to do opensource backups with deduplication, not how to draw a line with a ruler. Besides, anyone who tells you backups are simple is someone who has never had to do restores. – chris Feb 22 '11 at 03:09
  • 1
    this is exactly why you should try to keep things simple, because restores never are. – Chris S Jun 10 '11 at 02:43
  • ZFS is surprisingly simple to use, but you will need to understand the underlying operating system (FreeBSD, OpenIndiana/Illumos, etc.). There are appliances and distributions which can help (Such as http://www.ixsystems.com/freenas/). Backula is relatively simple when compared with enterprise backup software like NetBackup. – Stefan Lasiewski Jun 17 '11 at 21:01
2

I have used backuppc with good success on a range of windows linux and mac machines. http://backuppc.sourceforge.net/ http://www.zmanda.com/backuppc.html (info on De-Duplication)

Nick Downton
  • 492
  • 2
  • 13
1

Here few links for de-duplication of top of my head:

  • OpenDedup - de-duplicating file system that works on Linux and Windows.

  • s3fs - de-duplicated file system backed by Amazon S3 service.

dtoubelis
  • 4,579
  • 1
  • 28
  • 31
1

I ended up going with synbak http://www.initzero.it/products/opensource/synbak/ as it does the hardlinked copies using rsync and can pull them from remote servers unlike rsnapshot.

I'm also looking into using something like rwsync or deltacopy to setup rsyncd on windows machines so I can use synbak for windows machines as well.

user9517
  • 114,104
  • 20
  • 206
  • 289
user71173
  • 133
  • 1
  • 1
  • 3
1

You now have Obnam which does that also, and much more !

Anthony O.
  • 674
  • 1
  • 5
  • 13
  • While I love obnam, the [developer has dropped support](https://blog.liw.fi/posts/2017/08/13/retiring_obnam/) as of August 2017. – Andrew Domaszek Nov 08 '17 at 12:16