2

I've always been curious. It may be a stupid question but what if you deleted a file then filled up the SSD or HDD. Could your deleted data be recovered or is it physically impossible? I know Windows when you delete, you are actually just removing the headers from the files and waiting for the bytes to be overwritten (which means it still remains there).

But is it possible to recover deleted data (if not, some) from a full SSD or HDD (being that new files have overwritten the old data)?

forest
  • 64,616
  • 20
  • 206
  • 257
Lkvin
  • 29
  • 1
  • 2
  • 4
    technically this is weaker than to secure deletion of the file, as not the full file might be overwritten, but apart from that equivalent. And possible duplicate of: http://security.stackexchange.com/questions/35725/overwriting-hard-drive-to-securely-delete-a-file#35729 – Paul Apr 30 '16 at 23:07
  • 1
    I'm voting to leave open because @forest's answer here is almost a canonical answer for this type of question, and I hope other questions in the future will link here. – Mike Ounsworth May 01 '16 at 23:57

2 Answers2

4

There are three main types of storage drives. Hard disk drives, solid state drives, and solid state hybrid drives. Each of them behaves differently.

Hard Disk Drives

When you delete a file, you unlink it, effectively telling the filesystem that the sectors in which the file resides are no longer in use and are free to be overwritten. When this happens, the file is said to be present in unallocated space. Filling up an entire drive, in theory, will fill up every sector, including the ones which the unlinked file resided in, overwriting it. This is true for hard drives in most cases (rare exceptions include when a sector on a hard drive goes bad, and the hard drive is unable to write to it, causing the damaged data to be stuck in place). In general, you can assume that a single overwrite of any given sector is sufficient to destroy the underlying data. Random data is often said to be better than zeros. If you are on Linux, you should probably avoid using /dev/urandom as it is very slow. A very fast and sufficiently random alternative to create a random file to fill up all your free space would be the following:

# key=$(head -c 16 /dev/urandom | base64)
# sync
# openssl aes-128-ctr -nosalt -k $key -in /dev/zero -out /fillfile
# sync
# rm /fillfile
# sync

It's important to do this as root because some filesystems, such as ext4, have a certain amount of the filesystem reserved for the root user only. On ext4, this is 5% of the entire drive space by default. This is designed to prevent a regular user from filling up the entire drive and preventing daemons running as root from writing their important log files to the disk. However it also interferes with filling up the entire disk, so you must create the random file using root.

You can test if your drive has any bad sectors with the smartctl command. Install the smartmontools package, and run the following:

# smartctl -A /dev/sda | grep Reallocated_Sector_Ct | grep -o '[0-9]*$'

This should output the number of bad sectors on your drive. The first command lists a bunch of data about drive errors, the second command singles out the error related to bad sectors, and the third command displays only the last number, which is the raw value of relocated (bad) sectors in the drive's lifetime. In most cases, this value will be 0. If the number is higher, some data may still be left over on your drive. In theory, this could contain sensitive information. Each sector's size is typically 512 bytes (or 4096 bytes on some modern drives). If the number is very large, then you've got other problems to worry about, because your drive is about to fail soon, and you should probably back it up and wipe it before it dies on you. Note that this issue applies to solid state drives as well.

Solid State Drives

SSDs are a bit different. They contain an extra area called overprovisioning space, which is not accessible by the operating system. Due to wear leveling, filling the entire drive may not actually delete all your previous data, and files in unallocated space may remain in the overprovisioning space. Each time you fully fill up an SSD, you have only a statistical chance of overwriting any given sector, and there's a small chance that the sector will not be overwritten, and will instead be part of the overprovisioning space. Now, you can issue the TRIM command to some SSDs, which tells the firmware to erase the contents of all unallocated space. However, you have to trust the firmware to do it properly, and many do not. Additionally, there is no research on whether or not TRIM destroys data in a way that ensures it cannot be recovered with specialized hardware, as far as I am aware, even if you could trust the firmware to erase cells promptly.

(Since I wrote this, there is evidence of data being recovered from erased flash memory, even after multiple writes.)

On Linux, you can mount your filesystem with the discard option, or you can periodically run the command fstrim / (with / replaced with the mountpoint of any SSD you want to TRIM). Using fstrim has less overhead. Doing this is good for performance, and also helps make sure that data which is deleted is harder to recover. If you use disk encryption, you may want to avoid using TRIM however, for reasons this article explains quite nicely.

Solid State Hybrid Drives

SSHDs are hard drives which contain a small (several GiB) built in solid state drive which they use for caching reads and writes in order to speed up the drive without being as expensive as a full solid state drive. I don't believe you can issue the TRIM command to these drives, or even control the solid state component of these drives at all. The behavior of these drives is entirely dependent on the firmware of the drives, and is entirely unpredictable. Such drives could easily send all consecutive writes to the hard drive platter and only send random writes to the solid state drive, causing a large file used to overwrite the drive to overwrite none of the solid state drive at all, leaving plenty of sensitive data there. I would not use these drives if you want to be able to overwrite files at a whim.

Instead, if you need to hybridize an HDD and SSD, you could buy a small caching SSD and a larger HDD, and combine them using software such as bcache, which is far more efficient than crappy SSHD firmware, but also easier to wipe data off of if you need to. To wipe data off of them, you can disconnect the caching drive, essentially turning the system back into a regular HDD, then securely erase everything from the SSD, fill the HDD with a large file, and reconnect the SSD and turn it back into caching drive.

In summary, it's not easy to securely overwrite data by filling your hard drive with a large file. There are so many caveats to be aware of. If there is any chance that sensitive files may ever touch your hard drive, you should use full disk encryption. That is the only way to ensure your data is secure.

forest
  • 64,616
  • 20
  • 206
  • 257
  • There's a SATA command to do a full disc wipe. It's probably your best option in case of an SSD. – Sebb May 01 '16 at 00:50
  • There's one problem with the hard-drive-part of the answer. From a tinfoil-hat POV, the write **might** not fill up the entire sector. E.g. let's flood the entire storage with 1-byte long files (extreme) case and assume 512-byte sectors. Obviously we've only destroyed < 0.2% of the data. – Paul May 01 '16 at 00:56
  • @Sebb The OP was specifically asking about deleting a single file by overwriting unallocated space with a large file, so my guess is he does not want to obliterate the entire contents of the drive, otherwise I would have suggested that. – forest May 01 '16 at 01:58
  • @forest That whole talking about full overwriting distracted me. You're right, my bad. – Sebb May 01 '16 at 02:01
  • @Paul That's why you use a single large file, not a whole bunch of tiny sparse files. I don't think you'd have to worry anyway, because you'd run out of free inodes before the drive would be "full", so you'd know something was wrong when you'd see 7% space used yet no more files were being created, at least assuming you're using a modern filesystem like ext4 which starts out with a fixed number of inodes. – forest May 01 '16 at 02:03
  • @forest just wanted to point out that the solution of overwriting the entire drive **might**' have some flaws depending upon the approach that is used to do so and the platform. The example was extreme and should rather show a concept than an practical approach. The file may just as well be larger. If it doesn't precisely fill all sectors it's written upon there's still some data recoverable. – Paul May 01 '16 at 02:08
0

This turn out to be a more complicated question than most people realize. In concept you're right that if you delete (ie free) the file and then fill the disk, the file should be unrecoverable - in practice it's a little more complicated than that.

Just to avoid typing, I'll link to a previous post of mine on this topic. You should also check out wikipedia/data_erasure.

On that wikipedia page under the section Standards you'll notice that government standards from the 1990's for "secure deletion" required multiple rounds of overwriting for the data to be considered "unrecoverable". This is because magnetic drives write data in bands and in the 70's, 80's and 90's there was space in between the bands, which is where forensic analysts would recover the data from. Enough overwrites would eventually wear out that empty space. In that wikipedia article under the section Number of Overwrites Needed you'll see that since about 2006 hard drives have miniturized enough that there is no empty space between the bands anymore, so one overwrite is all you need.

You also want to consider that many modern HDDs are actually hybrid drives that have small SSD caches built-in that cache frequently accessed files. If you're simply doing writes to fill up the drive, your file may still be in this cache.

There's also the issue that as drives age (both HDD and SSD), they mark areas as "bad sectors" - meaning that those bytes will never get written to again. Whatever data was there when it was declared "bad" will be recoverable. I've heard some conspiracy theories that drives have algorithms to look for certain kinds of data (like private keys) and automatically flag this as "bad" so that it's not deletable.

Long story short: you're probably right that if you free a file and then completely fill the drive (either SSD or HDD), the file is gone. But, this topic is complicated, so I make no guarantees.

Mike Ounsworth
  • 57,707
  • 21
  • 150
  • 207
  • Not sure if the drive's memory cache matters at all. The larges enterprise drives I have seen have 128 MiB, but most tend to have around 8 MiB. And being RAM they vanish when the drive loses power anyway. The SSHDs on the other hand are a real problem. I had forgotten about those. – forest May 01 '16 at 02:54
  • If you're already wiping the drive, then all the RAM would contain is the random data being used to wipe the drive. If you're not wiping it and the RAM contains something sensitive (e.g. it's just about to write it to disk or it just read it from disk), then the hard drive would contain something sensitive anyway. In what situation can you think of in which the RAM would contain something sensitive but the disk would not? – forest May 01 '16 at 03:20
  • @forest I was wondering if writes would get cached or not, but I suppose it uses the RAM for buffering writes too. Fair, I'll remove. – Mike Ounsworth May 01 '16 at 03:25
  • It does cache writes. I suppose in theory some HDD firmware might dedicate different areas of memory for reads. I don't know why any would do that though, but given the poor quality of HDD firmware, I wouldn't be surprised if some existed which did that. – forest May 01 '16 at 03:36