44

If I were to archive data on a hard drive, unplug it, and set it on a (not dusty, temperature-controlled) shelf somewhere, would that drive deteriorate much?

How does the data retention of an unplugged hard drive compare to tapes?

HopelessN00b
  • 53,385
  • 32
  • 133
  • 208
Jim Hunziker
  • 1,802
  • 4
  • 17
  • 18
  • How long do you need to retain the data for? – hookenz May 13 '12 at 23:23
  • 3
    The answer to this question depends on whether we're talking about a mechanical hard drive or an SSD - though by the date on this question, I doubt SSDs were commonplace enough to warrant such clarification. – Nathan Osman May 16 '12 at 21:43

11 Answers11

38

Hard drives are unsuitable for anything other than short term archival storage. The problem is not one of data retention, it's the fact that stored drives have a bad track record for not spinning up.

Contrary to what Vilx- wrote, many of us know what happens to a drive that's been stored for 20 years - they won't spin up 90+% of the time. "Modern" drives are in fact worse than the drives of a decade or two back. CDs have been around more than long enough to know that the organic dye layer of an RW disc deteriorates surprisingly fast, especially when exposed to ultraviolet light (e.g. fluorescent lighting) and temperature fluctuations.

One thing that's often overlooked when comparing hard drives to tapes is the simple fact that backup tape technology is specifically designed to store data for extended periods. Hard drives are a temporary medium only.

Edit: The following was posted as an answer to a newer version of essentially the same question, so I'll merge it in here rather than have a second post

Hard drives are designed for relatively short term data storage. Quite apart from the mechanical issues there's the fact that magnetic media, and I mean ALL magnetic media, does deteriorate with age, even under absolutely ideal conditions. This doesn't require any external influences as magnetic particles are affected by others near by and in modern drives the density is nothing short of astonishing.

A drive in use has the magnetic strength regularly refreshed, either by actual writing to disk or by the drive's own low level system, which periodically reads and rewrites sectors. A drive in storage or even one that's simply powered down, receives none of that refreshing of the data.

As for how long a drive can be stored without becoming at least partially unreadable is an ongoing topic of debate and will certainly vary even across drives from the same batch, let alone different models or manufacturers. I personally would never rely on a device specifically designed for short term data storage for any significant length of time. Even tapes, which are designed for relatively long term data storage, should be refreshed ever few years.

With all that said, I don't know what your alternative options are. Gold layer CDs/DVDs are currently being claimed to have 20+ years of safe storage but I can also remember when the exact same claim was made for burnable CDs. Those early ones proved to have a safe life of less than two years.

John Gardeniers
  • 27,262
  • 12
  • 53
  • 108
10

The best answer re: archiving data is to move it to new media periodically and not to rely on any given type of media to "go the distance". If you want to see examples of "extreme" data archive maintenance read about CERN and their migration of data between tape "silos" (large robotic tape loaders with multiple drive and picker elements) of different tape technologies. (Those guys are generating a lot of data...)

Tapes last a long time but, as other posters pointed out, having a tape drive that is in mechanical working order, can still be attached to a computer, and has software support for the storage format on the tape is a challenge. You have to plan on moving data onto new medias as the old medias and their drives and software get old.

My strategy with my largest data-producing Customer (a court that records audio/video of trials and hearings) has been to write multiply redundant copies on LTO-3 WORM tapes of data, and to keep the data live on the SAN (but no longer subject to daily backup) so that it exists in at least 4 places at any time-- 3 of which are off-site. We've planned to move to LTO-4 when the current tape drives turn 5 years old (in 2 years). At that time we'll migrate all of the data on LTO-3 WORM tapes to LTO-4 WORM tapes. We'll also replace the SAN and migrate all the data there to the new SAN.

Evan Anderson
  • 141,071
  • 19
  • 191
  • 328
  • Good approach! One comment: Make sure that when you write the data you checksum it, and double-check it when you read it in to copy it down to the next generation of tapes. – Karl Katzke Aug 09 '09 at 03:47
  • We've been using the "duplicate job" functionality in Backup Exec w/ "verify" turned on to create our multiply-redundant LTO-3 WORM copies. I'd guess that we'll probably do the same with the LTO-4's when we get there. Fortunately, the files themselves are digitally signed by the recorders as they're created so that we can tell if there's been any modification or bit-rot. (Obviously we don't want bit-rot to happen, but it's good that the files themselves can tell us if there has been. It also prevents modification of "the record" w/o access to the private keys stored in the recorders.) – Evan Anderson Aug 09 '09 at 03:49
  • @EvanAnderson: Shouldn't the signatures be refreshed by regular time stamping? I mean, if the private key is *ever* compromised then *all* non time stamped signatures are useless. – Hubert Kario May 26 '12 at 10:32
  • @HubertKario: The signatures are a black box to me. The vendor who provides the audio recording hardware handles it and it's Not My Problem(tm). Relying on a private key to remain uncompromised for all eternity is definitely a bad idea, but nobody asked me how I'd handle it. I agree w/ you and wouldn't have done it this way if I'd been involved in the development of the audio recording software. I just admin the servers and make sure the data is maintained. – Evan Anderson May 26 '12 at 20:56
  • @EvanAnderson: Most people that work with crypto don't understand it, I'd be surprised if they implemented the signatures properly, let alone protect against non-repudiation. If you're the Chief IT guy it may be your obligation under law to ensure this (in EU it is). I'd suggest an audit and legal counselling. – Hubert Kario May 27 '12 at 13:46
5

Only because only optimists seem to answer so far, I have to put my fly into the ointment. (Note that I have zero experience with storing HDDs).

Will your HDD demagnetize over time ? What is the strength of magnetic field near it's storage ? (earth magnetic field, speakers, motors, nearby power cables, transformators, wireless phone chargers).

Will it's motor get stuck ? Does it have electrolytic capacitor which will eventually dry-out ? Will you drop it to a concrete floor some day ? Will you connect it wrongly, or put it's electronics touching some metal screw and it will fry ? And so on...

I assume the answer to all those questions is no. Are you Ok with relying on this assumption ?

On you spare time, read this:

....I immediately grabbed all of my stored hard drives and started finding computers to connect them to so I could see if my precious data was still there. Ten percent of the external Firewire drives would not spin up. During one Steely Dan tour, we recorded all of the shows using a pair of Mackie 24-track hard disk recorders. We filled 70 20-Gigabyte drives. None of them will spin up now.

Sandman4
  • 4,045
  • 2
  • 20
  • 27
5

One of the rules of thumb is that the smaller the magnetic domain, the easier it is to flip it. One of the reasons that spaceflight hardware is so less powerful than what we use down here is because high density electronics are much more vulnerable to things like cosmic ray strikes and high radiation. Long-term storage has similar concerns, as the amount of entropy required to corrupt information goes down as your feature size goes down.

What is the degradation rate? Hard to say, we haven't had these things all that long.

Archival over the production lifetime of your HD interface? Hard-drives will probably last that long deactivated. Archival over 20+ years? I'd be really leery of that without regular refreshes and migrations to new hardware.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
4

Since archiving digital data has been a subject of research for a while now, a number of papers and recommendations exist on this topic. Protecting digital assets for the long term by Joe Jurneke is a recent one trying a comparison between tape and HDDs for archival purposes. In summary, it makes the following points:

  • HDDs are subject to "infant mortality" - i.e. a new drive is has a much greater chance of failure (in the magnitude of 40-fold) of dying than after a burn-in-period of 3-6 months
  • HDDs have to be handled with greater care to avoid mechanical damage
  • HDDs are much harder to recover from mechanical damage, in many cases success is uncertain
  • HDDs were never designed as an archival system in the first place
  • HDDs are more subject to thermal influences on magnetism than tapes
  • production processes for HDDs change often and are specifically subject to bad manufacturing batches, making reliability assumptions difficult

With all this in miind, it is near impossible to answer the "how long" question.

Alternatives

Current research indicates that given the right storage conditions, phase-change optical media and magnetic tape are both a decent choice for an archival duration in the magnitude of 10 years. In fact, quite a number of manufacturers offer "archival grade" versions of optical media like DVD+RW or Blu-Ray which are claimed to last 30 years or more.

A "true" archiving solution which would last a period in the magnitude of 100 years is yet to be found.

Community
  • 1
the-wabbit
  • 40,319
  • 13
  • 105
  • 169
3

As there is a less easily replaced mechanical part to a drive, I would suggest that taps would last longer (if the tape real won't wind you can take the magnetic tape out and put it in another cassette more easily than you could move the platters of a drive if it stops spinning up or the heads refuse to move).

In either case there are three important things to do whether you use disks or tapes, assuming your data is important enough for the hassle (and presumably it is, or you would not be asking this question):

  1. make sure you keep at least two copies of any item of data on different physical media (separate drives, separate tapes, or one copy on drive and one on tape)
  2. test the backups regularly, to make sure the media still work and comparing the contents against stored checksums, so if one of the copies fails you can make another before it is too late.
  3. regularly review the media you have chosen to make sure that the technology to read it is still common (so if your equipment breaks it can be replaced easily) - as the tech starts to become more rare move the archives onto what-ever the newer media/adaptor standard(s) are.
David Spillett
  • 22,534
  • 42
  • 66
  • Tapes last very long. Too bad many companies no longer have drives that read the format of those old archives. I would suggest that it won't be too far into the future that you'll have IDE drives that don't mate with the SATA connectors the next generation of admins will be asked to retrieve data from... – Bart Silverstrim Aug 09 '09 at 00:05
  • Good point. I've added a third point to my "things to do what-ever medium you chose" list. – David Spillett Aug 09 '09 at 00:24
2

Once you will properly protect it against electro-static discharge I'd say you can sit down, relax and wait for the materials to decay. That if your karma is high and no external force will start acting up against your luck. If your karma is good then you will probably die before the disk does.

user237419
  • 1,663
  • 8
  • 8
2

I trust disks for precisely the length of their warranty, not a moment longer. For the kinds of disks I use that's almost always 5 years and I'd be happy to use a disk that's been sat around for 4.5 years for a further 6 months or so.

Chopper3
  • 100,240
  • 9
  • 106
  • 238
1

If you are not using the hard disk, there should be reason for it to fail given that it is protected against bad environment factors like dust, humidity, etc..

Khaled
  • 35,688
  • 8
  • 69
  • 98
  • 2
    +1 Toss it in a zip-lock baggy with a couple desiccants, should be good for 30+ years, assuming you've got the tech to power and interface with modern disks that far in the future. The silica-gel packets that most stuff gets shipped with are great desiccants, but usually need to be reactivated (the nicer ones come with directions printed on them; basically just cook in an oven at 225°F/110°C for an hour). – Chris S May 13 '12 at 14:45
  • 2
    @Chris S - The tech to power the interface is a good one. But, I doubt your motor will spin at 30 or maybe even 10 years when sitting. It similar to what happens when a car engine sits with oil in it for a long period of time. So for long term storage they generally drain all the oils out. – hookenz May 13 '12 at 23:23
  • @Matt I've had the freezer trick work surprisingly well on hard drives. As with abusing what any technology was meant for, YMMV. – Chris S May 14 '12 at 00:31
1

Don't expect too much.

For example, several years ago I had Seagate drives that didn't start if they stood around too long. The problem was in the composition of the bearing grease - which would slowly harden over time.

The solution in this time: switch on the computer and start gently knocking the drive from the side several times with the (wooden) handle of a screwdriver. If you had luck, the platter would start to spin again - if not, ...

Don't take anything for granted that has to do with hard drives.

Regards

rbo

Addendum: Another discussion on hard disk archive usage

Community
  • 1
rubber boots
  • 145
  • 7
  • so it seems tapes are still the way to accomplish long term task... opinions? – Arthur70 May 13 '12 at 19:56
  • @Arthur70: this is like the Dinosaur-DNA problem. Even if you got the DNA error free from somewhere, you don't have the machinery (organism) that is able to replicate or develop it for you. This would mean you need the whole trail of devices and decoders in good working condition to retain your data to your contemporary context after "enough" years. – rubber boots May 13 '12 at 20:30
1

My guess is that it should be at least as good as tapes. Perhaps even better, because the magnetic disk itself is in a contained environment and magnetically shielded.

However, I'm pretty sure that nobody really knows what happens to a HDD after, say, 20 years of unuse. It's the same thing as with CD's and DVD's (the archival-grade ones) - they simply haven't been around that long. Well, OK, hard drives were there 20 years ago as well, but comparing them to hard drives of today is pointless.

Vilx-
  • 791
  • 4
  • 13
  • 25
  • 1
    Pointless comparison? Hardly! From a mechanical engineering side of things little has changed. – John Gardeniers Aug 08 '09 at 23:38
  • 1
    Really? I thought that a lot of things have changed since then, not the least of which is perpendicular writing technology; used materials, etc. – Vilx- Aug 09 '09 at 11:46