I am looking at the secure disposal of old flash drives, and one of the standards I have come across says that they should be overwritten three times. Why is one time not enough? I assume doing the process three times is more expensive than doing it once.
-
See here: http://security.stackexchange.com/questions/6133/best-method-of-retiring-hard-drives – paj28 Oct 30 '14 at 13:37
-
To the first 2 comments, I'm actually looking for the rationale not just the way people do it. I can see what the differences are and how it's currently done, I want to know why we do it those ways. – Arlix Oct 30 '14 at 13:43
-
3I think several studies have shown that a single overwrite, *if it is complete*, is sufficient to protect against anyone other than the NSA. And doing further overwrites adds little. – Hot Licks Oct 30 '14 at 17:25
-
1And see also [Secure wiping of EEPROM and flash memory](https://security.stackexchange.com/questions/37317/secure-wiping-of-eeprom-and-flash-memory) regarding flash. – Gilles 'SO- stop being evil' Oct 30 '14 at 17:29
3 Answers
Overwriting the data is either insufficient or useless, depending on how things are done internally by the device itself. Flash memory has a limited life, expressed in terms of read/write cycles. To sum it up, you can have one block of data full of zeros; bits can be changed from zero to one individually, but the reset to zero can be done only for a complete block at a time, and if you do it more than 10000 times or so, then the block fails down. Blocks have a size of a few dozens of kilobytes.
Thus, if Flash-based drives simply used a direct mapping from the logical sectors (as seen from the computer) to the Flash blocks, they would be slow and unreliable: each write of a sector over a previous value which is not all-zeros would entail reading the whole block, clearing it, then writing it back with the new sector value. Instead, Flash drives use wear levelling methods in which the writes get spread throughout the device blocks, so as to both minimize the number of required block clearings, and avoid having some blocks more used than others. Moreover, Flash drives often have more physical blocks than their nominal size, both to give some breathing room to the wear levelling algorithms, and to allow for remapping a few failed blocks, thus augmenting the drive lifetime when blocks begin to break down.
A consequence is that you cannot be sure, when you overwrite the whole disk, that you really wiped out all the data on the physical medium. In fact, if wear levelling is applied by the drive, then it is pretty much guaranteed that a single overwrite will not overwrite all traces of the data at the physical level, because there are more physical blocks than the logical total drive size.
Using three successive overwrites is an attempt at ensuring a more complete coverage. But it relies on some assumptions on how the wear levelling works, which may be unwarranted since there is no standard. Since the wear levelling is entirely handled by the drive itself, in its firmware, there is no reason for such algorithms to be standardized or even documented; one may even be so bold as to point out that wear levelling algorithms are part of the trade secrets of drive vendors, so they are unlikely to be all documented. Whether three overwrites are more than need, or, to the contrary, insufficient to ensure data destruction, is simply not known.
Moreover, if the drive remaps fails block transparently, then a failed block still contains mostly intact data, and since the firmware will thereafter avoid using that block, that data will remain in place until the end of times, regardless of how many overwrites you perform. That failed block is permanently out of reach of the host computer. (Note that this argument also applies to magnetic disks, but remapping of failed sectors is a rare event for classic disks, whereas it is very common in Flash drives.)
So the main benefit of the three overwrites is psychological. It is a very visible procedure that helps convince managers and auditors that security is happening, thereby justifying the corresponding budget line. If you want to really wipe out an old drive, burn it.
- 168,808
- 28
- 337
- 475
-
5Psychological - yes, but also a carry over from when it actually did do something useful. 20 years ago (possibly more) it was possible to gain information about the bit state prior to a wipe, so Gudradain's answer is valid for that time period. – Rory Alsop Oct 30 '14 at 14:02
-
@RoryAlsop Good to know that it is no longer the case. What change in the hardware so that they no longer leave a trace after a wipe? – Gudradain Oct 30 '14 at 15:54
-
1@Gudradain I believe the residual bit information issue was a problem prominent in magnetic tape. As tape drives became less popular with personal computer users, this issue became less of a concern. – Vilhelm Gray Oct 30 '14 at 16:32
-
2But 20 years ago, indeed we were not using Flash drives... However, even for magnetic drives, remanence has reduced a lot because information density has increased; so when the drive is writing a "1" it already gets something in the 0.6 to 1.8 range or something like that. The small differences that remanence relies upon have been exploited to store more bytes in the disk. That's how we get disks with terabytes of data nowadays. – Tom Leek Oct 30 '14 at 18:28
-
2@VilhelmGray Tape, yes - but also hard drives. The read/write head is on an arm, moved by a "voice coil" or stepper motor in really old drives. Head positioning is good, but there's a margin of error. If you position a head a few micrometres to the left, you potentially leave original data on the drive surface (a few micrometres to the right). Multiple writes help ensure the whole drive is clean. Failing that, as per the answer - Industrial Incinerator. Have security personnel WITNESS the drives being loaded in, then sign them off as disposed. – Alan Campbell Oct 30 '14 at 23:47
-
Not only does wear-leveling make it hard to ensure you got all the blocks but it could actively fight you if the wear-leveler is fairly simple. Consider a wear-leveler that simply offers up the block with the most life left when writing something--what happens if you had one large file stored on the flash drive for a long time while other stuff was written and erased many times. Presto, a bunch of blocks with little wear on them, the wear leveler is going to hand them out repeatedly and ignore the other more-written blocks if it can. – Loren Pechtel Oct 31 '14 at 04:45
It's to remove any trace of the previous memory. Not an expert but here is what I understand.
Memory is stored in binary format : 0 and 1. So you can see your memory as a big array of 0 and 1 that we will call bits. Bits are not exactly 0 and 1, maybe 0.01 or 0.99 for example.
If a bit was 1 and you rewrite it to 0, it might be 0.02. On the other hand if a bit was 0 (0.01) before and you rewrite it to 0, it might still be 0.01. Hence, the previous 1 bit is leaving a trace that is possible to detect.
By rewriting it many times, you remove that trace.
Edit
This answer only apply to magnetic tapes which are no longer used in modern computer. Rewriting many times come from that time but now it's probably useless and only done by habits.
- 6,921
- 2
- 26
- 43
-
This. First look at magnetic drives: You read the magnetic level (pardon my lack of proper jargon), and if it's above some value then you read that as a 1. If it's below that value, you read it as a 0. However, if you have a very sensitive reader, you may be able to determine that a value just shy of your threshold USED TO be a 1, even if it's a 0 now. Let's go to Flash: Much harder to do (on its face, at least), but at some point of sophistication one can extrapolate a few possible previous values and then look for logical results. It's a bit like decrypting physics. – Smithers Oct 30 '14 at 20:59
-
2Is this really the case? Then would overwriting data with random bits remove all traces at once? – Question Overflow Oct 31 '14 at 02:50
It is a futile thing to do as explained well by Tom Leek, but luckily it also isn't necessary. The idea of overwriting several times is very old, the intent being that there remains a residue when overwriting data on a magnetic medium. Overwriting several times would (should) make lab recovery harder or impossible.
All modern drives, including magnetic ones, regularly perform transparent remapping, and solid state drives stack wear levelling on top of that, so you do not really have a chance of knowing if you really overwrite what you think, nor what remains on the disk.
That is the reason why you will always want to make sure that critical security-relevant data is in a non-swappable memory area. Once on the disk, you can't control its lifetime any more.
Now for the good news...
Other than conventional harddisks which add forward error correction to the data, flash drives encrypt all data (well, they should anyway, and most do that). This has, apart from being a cool marketing gag (the "uses AES encryption" sticker on the box), two effects.
First, some wear-levelling algorithms work more efficiently, since all data is "random looking". Overall lifetime is increased.
Second, you can toss the drive's key away (depending on whom you ask, this may be called a factory reset, a secure wipe, or bare metal format, or some other fancy name -- the manufacturer's setup tool will do it). Doing so will render all data on the drive, including sectors that have been remapped, unreadable.
- 5,001
- 1
- 19
- 26