29

Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.

If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?

For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?

Motivated
  • 1,493
  • 1
  • 14
  • 25
  • 1
    SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments). – forest Jan 11 '19 at 06:43
  • @forest - That is correct. I have updated the question. – Motivated Jan 11 '19 at 06:48
  • Confusing: Are you concerned with hard drives you are *buying*, or hard drives you are *disposing of*? – Harper - Reinstate Monica Jan 11 '19 at 22:03
  • @Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices. – Motivated Jan 12 '19 at 20:01
  • I just don't get why you care about someone else's security problem. And I have never heard of a non-profit that handled PII yet was so poor they had to scrounge computers. My nonprofits have no secrets that would warrant worrying about bad sector leaks, and if we did, that itself would be disturbing. So for us, the greater threat is data loss due to overuse of security. – Harper - Reinstate Monica Jan 12 '19 at 22:18
  • @Harper - The non-profit has for example limited approved funding for technology spend so it's is heavily dependent on donations, reusing existing hardware or limiting investments to essentials. Hence if additional storage was a requirement, the organization would default to the cheapest solution. The non-profit works with family violence, medical services, etc hence the sensitivity. If a leak occurred, it could result losing funding for example. You would be surprised with the requirements to meet compliance with limited funding especially in developing countries. – Motivated Jan 12 '19 at 22:37
  • I worked for a 56 million dollar non-profit. I couldn't get them to order new laptops to replace 7 year old, broken hand me down laptops. If I needed a new HD to replace a broken one, I would have to pull it from a junk pile computer that previously had medical PHI. So it would be my security problem as an IT worker and HIPAA compliance analyst @harper – cde Jan 12 '19 at 23:38
  • @cde - That's interesting. What approach did you take in the handling of sensitive data? – Motivated Jan 12 '19 at 23:42
  • @Motivated Bitlocker software level encryption at minimum, hardware level encryption drives if possible, DOD wipes for older disk drives (Not that the DOD 7 pass standard is effective but I couldn't convince the not very IT IT manager about that leading to complaints that the 7 passes HE MANDATED were taking too long), and ATA secure erase for SSDs. We also paid for "keep your drive" warranty for computers so if the disk or other hardware died under warranty, Lenovo just shipped a new one without needing the old one back. – cde Jan 12 '19 at 23:51
  • If you can, simply say the drives are broken/unusable (or break them) to force a new disk be ordered. YMMV on that but sometimes rank and file IT has to force an unwilling manager's hand. – cde Jan 12 '19 at 23:52
  • @cde - What solution was used for hardware encryption? – Motivated Jan 13 '19 at 02:27
  • Off the shelf samsung or other brand SSDs. – cde Jan 13 '19 at 02:36
  • 1
    $56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?" – Harper - Reinstate Monica Jan 13 '19 at 19:41

5 Answers5

51

Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.

If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.

There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Secure Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.

Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.

Glorfindel
  • 2,235
  • 6
  • 18
  • 30
forest
  • 64,616
  • 20
  • 206
  • 257
  • Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/88191/discussion-on-answer-by-forest-does-the-destruction-of-sensitive-information-lim). – Rory Alsop Jan 12 '19 at 19:22
13

Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.

Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)

For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.

Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.

John Deters
  • 33,650
  • 3
  • 57
  • 110
  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting. – Motivated Jan 11 '19 at 07:34
  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach. – John Deters Jan 11 '19 at 13:32
  • Deter - To clarify, what are you referring to an unspecified level of assurance? Do you mean the inability to sufficiently assure that there may be data leakage as s result of wear leveling? Secondly, to what extent can meta data associated with the encryption leak if taking into account wear leveling? I would have thought that overwriting a non-flash device would not result in any residual data. Additionally, what would you consider to be a planned approach? – Motivated Jan 11 '19 at 16:44
  • 1
    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives. – John Deters Jan 11 '19 at 16:55
  • Thanks. Do you mean to say that cryptographic solutions overcome bad blocks such that data is non-recoverable? – Motivated Jan 11 '19 at 17:01
  • To clarify, i am not attempting to rationalize disk wiping. I am seeking clarity if given the options to encrypt and wipe, do non-flash devices offer a higher level of assurance with all factors being equal. Additionally, if overwriting isn't a consideration, why are there overwriting standards (https://en.wikipedia.org/wiki/Data_erasure#Standards)? – Motivated Jan 11 '19 at 17:07
  • 2
    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned. – John Deters Jan 11 '19 at 17:13
  • Reliable access isn't the opposite of security- [it's a key aspect of security!](https://en.wikipedia.org/wiki/Information_security#Key_concepts) Perhaps that should be rewritten to specify that disks aren't designed for *confidentiality* first? – 8bittree Jan 11 '19 at 23:17
  • 1
    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it. – John Deters Jan 12 '19 at 04:22
  • @JohnDeters no method is guaranteed. You can shred the disks but a scanning electron microscope and enough time/effort/money can still recover it. It's all about how difficult you can make it. NIST standards, NSA testing (Though you can't really trust those guys) and today's Terabyte storage make a single pass overwrite good enough for "Clean" level sanitation. – cde Jan 13 '19 at 00:01
1

Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.

Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.

However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.

With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.

Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.

Micheal Johnson
  • 1,746
  • 1
  • 10
  • 14
  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs. – John Deters Jan 11 '19 at 13:37
  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance. – Motivated Jan 11 '19 at 16:49
  • `you don't need to erase the actual data, just the encryption keys.` While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard. – cde Jan 13 '19 at 00:11
1

Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).

Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.

Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.

Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.

There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.

Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.

Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.
And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!

Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.

Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.

So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.

Damon
  • 5,001
  • 1
  • 19
  • 26
  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts. – gnasher729 Jan 13 '19 at 11:35
  • @gnasher729: Well the _encryption per se_ isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical `if(trial == true) {...};` code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for _disposing_ this doesn't matter. Key erased is erased. – Damon Jan 13 '19 at 22:51
1

The way ssd drives handle their secure erase is by encrypting the drive and then securely overwriting the encryption key. Technically the data is still there it's just theoretically not possible to access it. I used a more manual way.

I securely erased a couple thousand SSD drives when I worked for $LARGE_INTERNET_VIDEO_COMPANY.

NOTE: these drives were provisioned with 20% empty space using the ata max_lba command to prevent the pathological case of thrashing the same blocks as you fill up the drive.

I used the linux command-line tool called shred. The default is 3x random re-write (perfectly safe as far as my research showed). I also added another pass with zeros. Something like

shred -z /dev/sd$DRIVE_NUMBER

Making a last pass with zeros is nice because it's dead simple to verify it worked. If you sum any given blocks you should get zero.

You could create a linux live usb boot device. With one command and a few minutes you can securely erase the drive of any machine. (and you can verify it) Alternatively you could do this on any drive you plug in.

jorfus
  • 441
  • 3
  • 6