Event triggered backup systems -- what are my options?

Question

Background - I work in a corporate environment where files on network shares are constantly being modified. Created, deleted, overwritten. Of course with hundreds of users, you can imagine the odd time where people accidentally delete the wrong files or overwrite them at the wrong time.

What I am searching for essentially is an event-based backup system. Something that can monitor the entire filesystem for events and create snapshots of files when modified. There are libraries out there that make this possible like inotify (Linux), so I'm wondering if it has been put to use for a server backup system, whether it be Windows or Linux based (there are inotify ports for Windows as well as the Java platform). We are currently using Symantec backupexec which works wonders, the only problem is I cant be having backup after backup running constantly on the servers, especially during peak hours which is when I'd need it the most. I find an event based backup system would be much more efficient as it would only take action upon file modifications, as opposed to constantly snapshotting the entire disk at once. In our current environment, users can "accidentally" obliterate files only hours after they are created. Having our backups run once daily after hours doesn't help me restore these deleted files.

Essentially a system that can keep a desired amount of snapshots on a per-file basis would be ideal. Possibly the last 5 - 10 copies of a file.

Storage is not an issue, I have multiple servers available ranging from 2 - 10TB space. We've also recently purchased a SAN system with 60TB space which is itching to be utilized. We are only dealing with small word/excel/pdf documents as well.

Any help and insight is greatly appreciated.

Answer 1

I think what you want is a versioning filesystem, which is simply a filesystem that keeps multiple versions of old copies of the file. I've not tried any of them, but a quick survey shows that NilFS could do what you want.

Answer 2

Windows kind of comes with something like this baked in. It's called Previous Versions in the GUI, and uses Microsoft's VSS to do the hard work. It's not exactly what you're after, but it has the nice side effect that you don't have to restore files from backup so often as the users can often do it themselves.

It's turned on per volume, and a scheduled task runs at intervals you define and takes snapshots of files that have changed since the last time the task ran.

This can help in the situation you describe where for example a user creates a file at 11AM (after last nights backup) and deletes it at 4PM. If for example the scheduled task ran at 1PM, the user can retrieve the file themselves by right clicking the folder it was in and selecting the Previous Versions tab.

This article talks about it in Windows Server 2003, but it's the same through to Windows Server 2008 R2.

Answer 3

If you have an inotify-enabled Linux kernel look into the incron/incrontab utility:

http://inotify.aiken.cz/?section=incron&page=about

It uses the inotify system to run commands that you configure.

Answer 4

For windows, you could look into Backup Exec's continuous protection services. It should allow you keep copies as they change, and stage them out over time. We have a VP assistant who keeps deleting files from our shares, and we have been able to recover everyone of them.

Answer 5

You could ask your Symantec salesdroid about NetBackup RealTime, which appears to do what you want by getting copies of all writes to the backed-up LUNs.

Warning: as with other NetBackup-family products, I expect an eye-watering price.