Real-time offline folder-to-folder backup application needed (Windows)

Question

I recently started using Intel Matrix Storage RAID solution that allowed me to use my 5 1TB drives for two RAID volumes. First one a 1TB RAID 0 striped across all 5 drives and second one a RAID 5 across the rest of the free space on all drives (around 2.85TB usable space).

The RAID 0 I use for OS, applications and games while the RAID 5 I use as a more-permanent type storage (photos, etc). Now I do realize that running the OS and applications on RAID 0 across 5 drives is very dangerous, which is what brings up the following question.

Is there a reliable freeware realtime backup application that can backup a set of folders from one drive to another drive (no online backups needed)? I've already tried a few (Mozy, Yadis, Comodo Backup, GFI Backup, Idoo, Crash Plan) but none meet my requirements:

• Low CPU and RAM usage.
• Realtime Backups - as soon as a file is modified in the source folder, it is added to the backup queue which will be processed with the lowest priority when the CPU is idle. This backup queue should persist in cases of computer restarts (ie: the source and destination folders should always have the same set of files, except for the ones waiting in the backup queue).
• Incremental Backups - if only 10 bytes changed in a 1GB file, the app should only copy those 10 new bytes.
• Ability to back up locked and opened files (some apps, like Yadis, can't back up critical files like browser favorites).
• Ability to run as a service (no need for any user to log-in to have the app started).

Optional requirements:

• Compression of the destination into a well-known format (RAR, Zip) that can be directly read without the use of the application.
• Preset source folders (such as Browser Favorites, Game Saves, Application Settings, etc).

The idea is to use RAID 0 array as "semi-persistent RAM-like" storage which in case of a failure can be quickly rebuilt by reinstalling the OS, apps and games and copying over the settings, saves, favorites from the RAID 5. I'm also thinking of taking this RAID 0 as RAM idea to the extreme with SSDs (as soon as we get some nice 6Gb/s SATA III SSDs out there), where a couple of SSDs chained in RAID 0 will work as yet another semi-persistent cache layer sitting between the RAM and the HD.

I'm just hoping there already exists an application that satisfies these requirements... otherwise I'll have to write one myself, which I would prefer not to do.

Answer 1

I do not think you can write something that is going to be sufficiently fast enough to keep you happy, while consuming very little CPU and resources. The task of compression into rar, zip, or 7z format is CPU intensive, and disk intensive.

I suspect that a "nearly realtime" system that can almost keep up with your working rates would have to be very intelligent indeed.

If you can not live with an overnight full backup of your Raid0 system, then you shouldn't be using Raid0 at all, in my opinion. I used this exact configuration for my primary system for several years and found no noticeable performance benefit in Raid0 when I did it with a bunch of commodity 7200 rpm drives. Instead, I decided to go with a single Velociraptor drive, and a second drive to back it up to. Lower MTBF risk, and higher performance for the same dollars, and in my opinion, no need for instant realtime backup.

In effect, IF you are trying to turn your Raid0 into something as "safe as Raid5" without losing the benefit of Raid0's speed compared to Raid5. I do not believe it can be done. In fact, I could almost guarantee that Raid5 in-hardware compared to backup-in-software will result in a net loss of usable disk bandwidth that will make your complex system both more likely to fail, and lower performance, than a single Velociraptor drive running unencumbered by realtime backup or RAID.

Answer 2

I didn't see a mention of OS. If you are running Windows, have you looked at RoboCopy? It is free. It can be setup as a Windows Service (using SrvAny), and (after the intial mirror) runs fast and uses very few resources.

I have used it to mirror a few folder structures from one server to the other where it was running as a service. I didn't have super stringent requirements for real-time in that instance; but I believe you can either have it run continuously, or on a very tight schedule.

Might be worth a look...

Answer 3

Have a look at MirrorFolder - not free though.