How do you back up multiple terabytes of data?

Question

While asking this question about a home NAS, Matt Rogish questioned my backup strategy for all this encoded video, and I had no answer. It seems that burning DVDs would be very time consuming. Also, do burned DVDs still suffer from degradation after n-years (where n is some number in the future)?

For those that don't want to read the other question, I currently have just under 1 TB of data and have estimated that I'll finish in the 1.5 to 2 Tb range, but may need more in the future. The data is primarily h.264 encoded video hand ripped from my DVD collection.

Answer 1

There are some studies here that look into the long-term storage of information on DVDs. DVDs do end up suffering from degradation eventually - according some of these articles, you'd want to go gold-plated.

Personally, I would buy a few harddrives and use those for backup. You could even get a NAS that supports either RAID or JBOD (Just a bunch of disks) configurations and back up your data that way.

Edit: You might want to keep an eye on this question: What are my options for a multi-Terabyte home NAS?, as the devices they're discussing are what you could use.

Answer 2

You have two basic approaches :

• the easy way is to buy 1 or 2TB eSATA drives, and use these as backup devices. It may be good enough if you don't need to back up every day.
• slightly harder : buy some used tape drive. LTO-2 should be pretty cheap now, and enough to backup a couple of terabytes.

I personally backup to disks. However the backup partition are offline (unmounted) when not in use.

Answer 3

I bought three identical external disk drives, copied my data onto all 3, then plugged one into my HTPC, one into my home office PC, and one into the PC at work. I use Windows Live Mesh sync on each machine to keep them automatically in sync. The two computers at home sync over the LAN, and the office one gets updated eventually.

No reason this shouldn't work with one PC at home and one at work (or a friends), and you can always do it with more than one drive too.

Answer 4

First of all, i probably would trust DVD/bluRay rather than HDD for long term archival storage. HDD has electronic and mechanical components and may fail even if you just let it sit unused for years: moisture, magnetization, etc. You can soak a DVD in water and it'll still read. If you're paranoid, burn more copies.

With several TB of data it's seems crazy to burn it all in DVD/bluRay, but your application is very specific: video. Have you consider using higher compression for archival purpose only? If you're ripping to divx/mpeg4 now, switch to h.264 at a lower bit rate. If you're using h.264 now, use h.264 AVC, downscale the resolution, reduce I/b-frames, downmix AC3 to 128kbps stereo AAC, etc.

Even DVD will look like VHS to us in a few years, so will bluRay. I'd be willing to sacrifice a little quality to make archiving easier and smaller, when the truth is that by the time you need that backup copy it'll look like VHS whether it was native mpeg-4 or h.264 AVC because technology has processed 10x already.

How many DVDs do u have and how are you encoding them now? At 1GB a piece compressed you need 1000 DVD to fill a 1TB drive.

If I were you I'll compress the hell out of those movies if i want to keep and burn them on DVD or other cheap redundant offline media or upload them to the cloud, but keep the higher quality rip online in hdd for instant viewing. Majority of them are not gonna be the action packed blockbusters that need high resolution and bitrate. And for the blockbusters, use higher bitrates for them but still keep a lower bittate copy offline for storage.

If that all sounds too complicated, it is. It makes you ask yourself the tough question, what data is more important than the other? just how much quality do you really need from a backup copy? And how much money and time do you really want to invest in a bit-perfect copy for everything? If you want to take the easy way out then just mirror everything on RAID. And if even then you wanna make the argument that RAID is not a real backup, that's just digging a deeper rabbit hole for yourself.

For music I have a simple painless backup strategy, ie. none. My primary library is on one hdd. I sync my 2 iPod and iPhone with it. If the library hdd dies, I still have the music I have on my iPods, which contain the most important selections of what I like and listen to the most anyway. The rest I'll lose. And I'm fine with it. I might get a backup drive for my music someday, but it's not a priority for me to backup "everything".

Just that we live in a world of bit-perfect copies doesn't mean we have to make a bit-perfect copy of everything we have. Prioritize your movie library, some you can stand to lose, some you can keep a low-bit rate copy. Even for the important movies, chances are you'll buy the bluRay version in a few years, and then the 3d version, and so on. Their future redundancy is already built-in to you.

And lastly, if raid is not real backup, then neither is a hdd storage tower elsewhere in the house. It's still a bunch of spinning hdd. They WILL fail in 3-5 years. It's a statistical inevitability. If you buy 2TB for backups, that's extra 2TB for you to upgrade every 3-5 years.

Just remember, all these DVD quality video will look like VHS to you in 10 years, and you can stream most of them from NetFlix for $15/mo, TODAY, already cataloged for you, and some even with 1080p.

Answer 5

Mozy. $4.95/month you don't have to worry about fires or theft