How to erase as much as possible an SSD without ATA Secure Erase?

Question

The question is almost completely answered. However, more details are still needed. See Update 2 down here.

I've learnt that the ATA Secure Erase is uncorrectly implemented in SSDs (sources are down here), but I'm still willing to find a way to erase as much as I can on them.

What I intend to wipe is the whole SDD, which includes:

• The cells that users may access
• Bad/unmapped/corrupted sectors
• The over-provisioned space
• The trimmed cells
• The Device Configuration Overlay (DCO)
• The Host Protected Area (HPA)
• And everything I've forgot

I know that encryption is the best way to simulate a limited "Secure Erase", but before doing that for my new datas, I want to at least make a single pass in order to wipe as much as I can of the old ones.

As I know so far:

• ATA Secure Erase: Not reliable, still wipe correctly HPA or DCO ?
• dban: Do not erase remapped sectors, nor HPA or DCO
• nwipe: Same problems as dban since it's a fork
• dd: Same as dban and nwipe, but also blocks everytime it meets a bad sector
• shred: Recommended for files, works like dban, may have issues with SSDs
• badblocks -w : Should check every sectors destructively, is it correctly implemented for SSDs ?

For now, the best I can do is a badblock -w.

So the question is: What tools can I use in order to erase data as much as possible on an SSD ?

The idea is, based on information I may not know -yet- but you do, to find the most suitable tool listed here, or any other tools not listed here.

Also, anything that may lead to correctly access or/then then delete an SDD's DCO or HPA is ok too. -> This was almost completely answered by @guest, but see update 2.

Same goes for remapped/bad/unnmapped sectors, trimmed cells and over-provisioned space.

Destroying the drive is not an option.

Sources:

• http://cseweb.ucsd.edu/~swanson/papers/Fast2011SecErase.pdf
• http://nvsl.ucsd.edu/index.php?path=projects/sanitize

Update: badblocks -w may not be reliable (https://lime-technology.com/forum/index.php?topic=23792.0), but I need to dig that up more, unless someone provide an answer here first, which I'm also interested in.

Update 2:

Now, the remaining thing I need to know is: Does the implementation of DCO and HPA respective erasure and disabling is effective -and not badly done like it is for ATA secure erase- ? Furthermore, a naive question here: Does disabling the HPA means this latter will get erased too ?

PS: Sorry if I don't answer right away, I'm working and travelling around the world -thus making this post related- and I often face time-squeezing business. But, I will definitely answer back for sure.

Answer 1

I don't know why you say that SSDs implement ATA Security Erase improperly. Modern ones implement it using SED (which uses an encryption key known to the drive and stored in non-volatile memory for all your data, allowing it to simply erase the key to instantly render data unreadable), while others mark all sectors for being TRIMed, which should effectively wipe the data when the garbage collector gets to them. I'm sure there are some out there which improperly implement it (it's not particularly uncommon), but the same applies to hard drives.

It is true that external USB SSDs and HDDs often have problems with ATA Security Erase, but this is due to problems with USB interfaces not playing nicely with the ATA standard. Additionally, USB flash drives (I wouldn't call those "SSDs") don't implement ATA Security Erase, due to having extremely simple firmware and very low-end microcontrollers (compared to SSDs and HDDs, which have very powerful processors, often dual core, sometimes even quad). An SSD is a full computer connected an FTL and an array of flash chips. A flash drive is a microcontroller with flash memory built in.

As for wiping with userspace tools, none of those will affect the DCO, and none of those will directly affect the overprovisioning space. Only using ATA Security Erase will you be able to erase all of that. It's designed to wipe reallocated sectors (as long as they are not too badly damaged), as well as the HPA. The DCO is just configuration data, so it won't contain anything now that it didn't contain when you got it from the store. If you really want to erase it, you can reset it to factory defaults using the Linux hdparm tool, which has a command to restore the DCO.

Note that badblocks -w writes a predictable pattern, and some SSDs will detect this and compress it, resulting in not actually writing the entire thing to the drive. For data destruction purposes, this is next to useless. You will need to write a random pattern to prevent this.

Just use ATA Security Erase. If you know for a fact that your specific SSD firmware has a buggy implementation, and you can't upgrade your firmware to fix it, then use hdparm to send the TRIM command to all sectors, and use dd (or a similar command like ddrescue or dcfldd) to write random data to your drive, and do this several times. Doing it several times is needed not because there is a way to recover deleted data, but because the overprovisioning space cannot be written to in one pass.

You won't be able to wipe completely damaged sectors. If the drive detects that a sector is taking longer than normal to return data, the data being returned doesn't match the internal checksum, or it has to retry too often, then it'll move it to a known-good sector and mark it as bad. It'll do the same if a sector has simply been written to more than a certain number of times (as they fail after a rather predictable number of erase cycles, often around 2000). The drive can be coaxed into manually marking the sector as good again in order to wipe it or put data onto it (which ATA Security Erase does, and which hdparm can do), but that's only if it was marked bad while it was merely losing health. If the sector was so badly damaged that it cannot return any data, and writes stall indefinitely/fail completely, then there's nothing you or the drive can do. The sector is unreachable without you (or an advanced forensic investigator) examining it physically. You can determine if there are any damaged sectors by using the smartmontools package:

# find out how many sectors are reallocated, and how many reallocations occurred
smartctl -a /dev/sda | grep -i reallocated

An example, if you cannot use ATA Security Erase, to wipe /dev/sda as securely as possible:

# find our max sectors (geometry = 91201/255/63, sectors = 1465149168, start = 0)
hdparm -g /dev/sda

# disable the HPA
hdparm -N p1465149168 /dev/sda

# trim every sector in the drive, from 0 to the max
hdparm --trim-sector-ranges 0:1465149168 /dev/sda

# create a crypt device with a random key, and overwrite it a few times
cryptsetup open -M plain -c aes -d /dev/urandom /dev/sda wipe
for i in {0..6}; do dcfldd bs=256k pattern=0$i of=/dev/mapper/wipe; sync; done
cryptsetup close wipe

# restore DCO to factory defaults (this can mess up the SSD, so do it last)
hdparm --dco-restore /dev/sda

But I would still recommend you use ATA Security Erase instead, if you can:

# initiate an ATA Security Erase which can't be disabled until it completes
hdparm --security-set-pass NULL /dev/sda
hdparm --security-erase-enhanced NULL /dev/sda

And next time, encrypt your data!

Answer 2

Short answer: Disabling the HPA and resetting the DCO only unhides previously reserved sectors on the drive, and does not involve any secure erasure itself. It only makes these sectors available to the system for later deletion by system tools.

Update 2:

Now, the remaining thing I need to know is: Does the implementation of DCO and HPA respective erasure and disabling is effective -and not badly done like it is for ATA secure erase- ?

You can disable the HPA (i.e. set its size to 0), and you can reset the DCO to factory defaults. Neither of these involves erasure. Unlike the HPA, the DCO cannot contain personal information (whereas the HPA may contain data that was deleted but not overwritten). The DCO is just a small amount of configuration data. If it was even possible to wipe it completely, you would brick your drive. All you can do is reset it to factory defaults. You don't need to worry about securely deleting it, because nothing will have been written to it since the day you bought it from the store.

Example of the information stored in the DCO, to show how little it actually stores:

# hdparm --dco-identify /dev/sda
/dev/sda:
DCO Revision: 0x0001
The following features can be selectively disabled via DCO:
    Transfer modes:
         mdma0 mdma1 mdma2
         udma0 udma1 udma2 udma3 udma4 udma5 udma6(?)
    Real max sectors: 1465149168
    ATA command/feature sets:
         SMART self_test error_log security HPA 48_bit
         (?): selective_test conveyance_test write_read_verify
         (?): WRITE_UNC_EXT
    SATA command/feature sets:
         (?): NCQ SSP

Furthermore, a naive question here: Does disabling the HPA means this latter will get erased too ?

No. Think of the HPA as a hidden partition. Configuring the HPA is akin to resizing this reserve partition. Disabling the HPA is setting the size to 0, allowing your operating system to now see and use the space that was once reserved. Once its disabled (set to 0), that previously-untouchable space can now be subject to erasure using system tools. But the only way you can interact with the HPA that matters is by setting its size.

Note that the DCO is also capable of hiding data by making the drive present itself as smaller than it actually is, although in a different way than the HPA does. Some vendors will intentionally reduce the size through the DCO in order to make one drive size, cripple it by limiting the size with the DCO, and sell it at a reduced price, selling the full-sized version (i.e. the version without DCO limitations) at full price. You can remove any potential limitations by resetting your DCO to factory defaults before performing any wipe, although beware that any low-level configuration changes to a drive carries risks

Regarding your comment on my first answer (I can't reply in the comments since this is a guest account), the paper you linked seems to be talking about older SSDs which improperly implement the ATA Security Erase standard. As I said, it's not particularly uncommon for the firmware to be written poorly enough to fail at implementing it. However modern drives all use SED, which negate the issue of flash cell memory remanence. Overall, a modern drive is likely to be more reliable, by far, than badblocks, by orders of magnitude.

A simple test would be to overwrite the drive with a pattern like 0xdeadbeef, initiate ATA Security Erase (with a SED-enabled drive), and check to make sure that the pattern is nowhere to be found. With this, you can know if it's been implemented properly. If the command returns almost instantly, and the pattern is gone, then SED has very likely been implemented correctly.

Answer 3

Assuming you can set up encryption before first use (and then lose the key to effect a wipe) it doesn't matter if you delete everything. It's completely irrelevant.

If you already have data on the device but it isn't encrypted, then it's too late for that. As you pointed out the tools available all have flaws (through wear balancing, blocking etc) so if the data must be securely wiped your only reliable option is to physically destroy the SSD.

So I'd suggest an industrial shredder (other destruction methods are available)