Ideas for a nearly 100% reliable portable data storage

10

3

I've had my share of broken storage media. During just last year, I've had:

  • two broken 500GB Seagate hard drives; one was new, broke completely, got different one - refurbished, but a little bit broken too (after running for a little while it just hangs)
  • new 1TB Seagate hard drive that is semi-reliable; usually works but sometimes it hangs and results in BSOD
  • my 16GB Flash Voyager GT broke just today; somehow a couple of hundred sectors are no longer usable; cannot read from them and all data that was there is lost

At one point in time I've moved all of my data (documents, programs, etc.) into one 8GB encrypted file container (TrueCrypt) and moved this file onto one of the most expensive pendrives in its range (the 16GB Voyager GT mentioned above) hoping it will prove to be 100% reliable solution. Unfortunately for me, it broke.

So now I'm asking you for a recommendation of a portable solution (max size is 2,5" hdd) that would be (nearly) 100% reliable. Which will not break. At least not easily. 10GB storage is enough.

Paweł Gościcki

Posted 2010-02-18T01:25:47.080

Reputation: 1 949

Did you get a replacement for the Voyager GT (I have one of these...)? Also, given how easy it is to lose a USB flash drive, this could never be close to 100% reliable. Do you have a backed-up master-copy somewhere else? – sblair – 2010-02-18T01:43:30.747

The Voyager broke just yesterday. I'm still trying to recover the data from it. I'm not afraid of loosing the usb stick. I never lost one. I have a backup copy of the data that was on the usb stick, but it's from 5 days back. So 5 days worth of data is lost. – Paweł Gościcki – 2010-02-18T10:19:15.077

Seagate used to be a good HDD mfg, but I have read quite a lot of bad things (like in reviews) of Seagate drives in the past few years, specifically their external drives. I used to like Maxtor until a bunch of their drives died. I loved WD and still swear by them for the most part, but like everyone else, their quality is also falling. – Synetech – 2011-08-25T06:22:13.850

Answers

7

If 10 GB capacity is sufficient, then a 16 GB USB flash drive is the best solution in terms of robustness and reliability. A (synchronized) second stick, however, will double your chances to get your data home safely. :)

Molly7244

Posted 2010-02-18T01:25:47.080

Reputation:

That's what I though when I had bought my 16GB flash drive. The one that broke yesterday. Synchronized second stick solution is hard to apply when you deal with one 8GB file. I would need to copy it at least daily and that means 10 to 15 minutes without the ability to use the data. That's way too cumbersome. I've already did some weekly backups from the USB to local hard drive "just in case". But the last backup is 5 days old... so some data has been lost. – Paweł Gościcki – 2010-02-18T10:29:32.197

I was thinking that maybe the USB drive just wore off. That TrueCrypt was making changes just to few sectors of the drive and it has just used all of the available write cycles. But that is probably not true. Here's the log from the data recovery tool (bbcopy): Copied 8000000000 bytes, 2048 unreliable, 5703946240 in bad blocks. Approximately 71% of the flash drive is damaged. – Paweł Gościcki – 2010-02-18T10:40:50.313

1a backup being too "cumbersome" or "inconvenient" isn't really an excuse, now is it? and yes, even flash drives sometimes break, especially when used frequently, the number of write access cycles is limited, if it fairly new and a quality product, you may get it replaced under warranty, but the data are your responsibility. – None – 2010-02-18T16:13:22.233

I would also say an USB should be the best solution. Dunno if you had just bad luck, but they should last nearly 'forever' (yes, they will wear out eventually, but not quickly ... a German magazine tried to dead-write an early generation USB stick with years of nonstop writing and it's still working). – Nicholaz – 2010-02-19T09:57:10.977

@Nicholaz Do you have a link? It's possible that the USB drive used SLC flash, whereas new drives will tend to use MLC flash (for higher capacity at a reasonable cost). SLC flash should provide approximately an order of magnitude more writes in its lifetime than MLC flash. – sblair – 2010-02-22T15:08:02.877

3

@sblair: http://www.heise.de/ct/artikel/ueberflieger-291740.html#literatur The Article is German (it's the German magazine with the best reputation far and wide), so yuo'll have to run it through a translator ... the relevant part is down below the reference list under "Flash-Haltbarkeit" ("Flash durability")

– Nicholaz – 2010-02-22T17:55:44.200

6

Have to wonder - why so much of your equipment is dying - environmental? Electrical(surge?) accident (drops?) or other? Your solution would have to incorporate protection against this.

Also, have you considered a remote solution - RDP into a system at home or possibly keeping things on the cloud? There are things like Microsoft Live Mesh.

Blackbeagle

Posted 2010-02-18T01:25:47.080

Reputation: 6 424

I've no idea why there's so much hardware dying in front of me. I did not drop any of the hard drives. This particular usb stick is hard to brake as it has a 2-3mm thick rubber casing. Fall from 10 meters would not break it. Probably. Must be some voodoo that so much hardware dies :) And I've moved recently, so that's not the environment's problem. I think. – Paweł Gościcki – 2010-02-18T10:22:42.540

Keeping my data in the cloud could be a solution. But I don't trust enough any of those big companies to keep my data unencrypted there. And encrypting data usually means one big file container. At least that's the way I'm doing it now. It's 8GB. So incremental backups are out. Pushing 8 gigs over the net on a daily basis would require a much better uplink that I currently have. – Paweł Gościcki – 2010-02-18T10:26:11.223

4

Don't go with physical storage. Backup your data with Dropbox or to a server that is managed. Online space is pretty cheap, and you have full redundancy if they do daily backups.

Josh K

Posted 2010-02-18T01:25:47.080

Reputation: 11 754

I have all of my data encrypted in one 8GB file container. I cannot push it over the net on a daily basis. I don't have uplink fast enough. – Paweł Gościcki – 2010-02-18T10:30:42.193

Plus the “cloud” is physical storage; it’s not stored on magical fairy dust, it’s stored on hard-disks (albeit enterprise-grade server drives that are supposed to be more reliable). – Synetech – 2011-08-25T06:19:16.567

2

If 10 gig is enough (and yeah, shouldn't it be?), borrow my procedure if you see fit : - 8 gig with all that data on USB-stick, hash and double/triple-check at will; one kept at each parent's house (this is of course something one has to re-adjust as one sees fit, in case no-one is to be trusted, two separate bank vaults would be my guess...hardcore though...)

Be ultra-prudent (parents do not live in same town as me, or each other (again....adjust as see fit))
and rigorous in regards to the what and the when and the where, stick to it; and you've got something that's kept me (a klutz of dimensions) knowing that my that is safe...

("that" seems to imply something....exchange with "my digital personification" or what-not...)

Morten Bergfall

Posted 2010-02-18T01:25:47.080

Reputation: 131

I'm using similar solution. I have many backups of this 8GB encrypted file (with all of my data) scattered around. Different houses, different media. Some online, some offline. The problem is that most of those are old backups. Updating them with the current data is the problem. There is just no way to keep all of them current. – Paweł Gościcki – 2010-02-18T10:33:31.190

most importantly...never forget that you're human, and it's never going to be a snafu free life ;-) – Morten Bergfall – 2010-02-18T17:26:06.147

2

From what I've been reading you say, Paweł Gościcki, your main concern is not "portable" storage, but "encripting", "up to date" and "trusting services".

There are several points I'd like to address here, giving opinions and ideas. Sorry I don't really know any good ready solutions for it. But I guess some of them are available and others you may have enough frustration-power to build on your own.

For the breaking devices, even if you changed environment, I'd still bet it's some electrical issue - though it should not be if you really moved to far away. Removable media is more susceptible to static energy. If you are so consistent into breaking those gadgets you should really consider measuring more yours. I would.

For the "up to date", If I'm not mistaken, Dropbox (or maybe another service like Moby or Carbonite) actually is able to sync your files on your network without going to their server FIRST, so you really should give it a try as it would not consume your bandwidth. I haven't tried, but I think it would keep your 8GB chunk file in sync in your network, then you could use a second machine to open it up and sync to external media.

Even if there is no such option you could encrypt each file individually, rather than making that 8GB chunk that can't be easily synced. Get some script to first encrypt your files when you throw them in a folder and second throw them already encrypted on the Dropbox folder, or whichever. This won't be easy, maybe there's a solution already out there, but it's a simple idea to go around your issue. Of course, you couldn't backup your whole system like this.

And for actually using more portable devices, if you figure a service to use, or maybe using some sync software, you can get a second machine running with it and syncing everything you plug on it. That way you only sync your machine once with a device able to replicate external media at will, and easily.

Personally I just don't care about encryption. So I use the cloud, pen drives and time machine. Much simpler. :)

Hope I could be of any help.

cregox

Posted 2010-02-18T01:25:47.080

Reputation: 5 119

1

Despite your bad experience with the Corsair Voyager GT, have you considered the Survivor series?

sblair

Posted 2010-02-18T01:25:47.080

Reputation: 12 231

Corsair is now on my "do-not-buy" list of hardware producers, so I'm not really keen on buying from them anymore. – Paweł Gościcki – 2010-02-18T10:37:21.040

1

If you're not happy with either hard drives or USBs (which you're not), maybe you can go with an alternative approach: tape backup ... they're not as common as they used to be, but they're still here, used, and some people I know swear on them.

alt text

Apart from that (if it's only 16GB) you can also always burn 3 DVD-RWs.

Rook

Posted 2010-02-18T01:25:47.080

Reputation: 21 622

Besides that he asked for something the size of a 2.5" HD, I also swear by tapes ... that is I start swearing as soon as I have to work with one. If you want something fiddly that doesn't give you the data back when you need it, use a tape drive. – Nicholaz – 2010-02-19T09:55:06.813

1@Nickolaz - well, listen. He doesn't want hdd's, nor usb's, nor sdd's. He doesn't want remote backup. He doesn't want ... I gave him this as a last alternative I could think of ... I've worked with tape drives up to 5 years ago (or so, something like that) and never had any problems with them. More realiable then one who doesn't know expects. ... – Rook – 2010-02-19T11:29:27.093

@Nickolaz - and tapes are relatively small, btw. – Rook – 2010-02-19T20:44:35.273

1

I don't think you will find something realistic that is more reliable than a USB drive, so in order to increase your reliability you have to implement some form of redundancy.

You mention having to sync 8 gigs of data each day - is all of that data really changing every day? Could you perhaps have separate encrypted volumes for stuff that rarely changes (no need to sync every day, perhaps every week or month) and stuff that changes daily?

Console

Posted 2010-02-18T01:25:47.080

Reputation: 2 552

The encrypted container is just one 8GB file. I'm thinking about maybe going with half-encrypted solution when only certain parts are encrypted and then rsync daily/hourly to some other location. Having two or more separate encrypted containers is too cumbersome. You need to mount them separately and that's just too much hassle. I was thinking that there may be some extra-durable solutions based on flash memory. Corsair Survivor seems like the best solution so far, although I'd rather not go with Corsair this time. – Paweł Gościcki – 2010-02-18T13:55:29.310

0

I would not go with HDD. I would use Flash Drive or something that you can drop on the floor without losing all data.

USB key is the best. You can also have USB Key up to 64GB. It's quite simple but simply the best thing you can have.

r0ca

Posted 2010-02-18T01:25:47.080

Reputation: 5 474

1I've had numerious drives dropped on the floor (inside computers or separate) and they still work fine. HDD's are not that fragile, like the commercials (now that the storage industry has sdd's <- something new to sell) would like us to think. – Rook – 2010-02-18T01:41:57.987

It just somehow all broke. I put the blame on bad luck. Problem is I don't really believe in bad luck. – Paweł Gościcki – 2010-02-18T10:37:02.757

0

I work at a university looking after the Sports School's IT issues. One fo the problems we had was that they were constantly trashing Portable HDD's. they get put in sports bags, camera bags etc and knocked about all over the place. We invested in 2 of the BUFFALO HD-PX320U2 and these have been going strong for ages now, we're about to pourchase another 5 we're that pleased with them. The capacity is higher than the specifications you've provided but thats never a bad thing if you don't mind paying a little extra.

Joe Taylor

Posted 2010-02-18T01:25:47.080

Reputation: 11 533

It's just a durable case for a normal hard drive. Rubber USB sticks are better at this area (esp. when you are throwing them around). – Paweł Gościcki – 2010-02-18T13:51:54.837

0

I would advise you to try Transcend StoreJet. I have 3 of these drives and they are so shock proof that they can surive 6-foot falls. That's for the exterior, and from my experience (of having owned 10 hdds from seagate, western digital, and all other famous brands), Transcend has never given me problems. Infact, all of my Transcend drives are TrueCrypt encrypted and never had issues with bad sectors.

Since you don't have a fast up link, I would suggest you to give Transcend a shot.

Jay

Posted 2010-02-18T01:25:47.080

Reputation: 151

Spinning media like hard drives are bad by design. They break sooner or later. And it usually is sooner than later. – Paweł Gościcki – 2010-02-18T13:49:24.497

0

How about making a RAID array of Flash drives? It seems like a simple enough solution, seeing your demands, and I'm sure there's ought to be a portable solution somewhere.

Without going into details here's a solution off the top of Google, though I haven't checked whether it'll port to other computers http://www.bigbruin.com/reviews05/thumbraid_1

SurDin

Posted 2010-02-18T01:25:47.080

Reputation: 418

0

If you can "break" your 8GB bundle back into individual files, you could consider Jungledisk

p.marino

Posted 2010-02-18T01:25:47.080

Reputation: 198

0

I think you have an abundance of replies concerning storage. You have had various types of devices fail on you. This is unusual to me. I would suggest you have all your computer needs powered through highly rated surge protection. That would include protection on everything including your printer and internet connection as well. You could very well be getting dirty electricity causing these failures or possibly some sort of static discharge.

MiKeM

Posted 2010-02-18T01:25:47.080

Reputation: 54