48
4
I have always just unplugged my USB drives from my computer for years and nothing bad has happened; but I heard that you are supposed to use "Safely Remove Hardware" before removing a drive. What does the "Safely Remove Hardware" program do (besides flush the output buffer)?
Is this recommended just to ensure that all data transmission to the device has stopped, or is there another reason?
Nate Koppenhaver
Posted 2011-11-15T21:05:47.173
Reputation: 3 523
74
This depends on what settings you have set for the drive. For instance, if you look in the device manager for the disk drive itself, you'll see something like this:
If you have the first option (quick removal) selected, then ejecting the drive merely unmounts the partition, and no longer allows programs to access the drive.
If you choose the second option (better performance), then ejecting the drive flushes any caches the OS has in memory, unmounts the filesystem (which may also flush a few buffers), and flushes on-device buffers.
Billy ONeal
Posted 2011-11-15T21:05:47.173
Reputation: 7 021
7+1 for noting that "it depends" on the config, and including a screenshot – Lynn Crumbling – 2011-11-15T21:32:29.457
Nice answer! Would you be interested in writing about this briefly for the Super User Blog? If so, let me know in chat or by responding to this comment.
– nhinkle – 2011-11-16T05:49:48.717This assumes you're using Windows of course... Mac and Linux machines don't have that option AFAIK, but will still complain if you pull out the USB stick before unmounting the drive. – calum_b – 2011-11-16T11:19:11.767
@scottishwildcat - Mac and Linux machines defer the handling of cache to the filesystem code. FAT(32, exFAT etc) filesystems by default sync writes every 5s, NTFS(rw - fuse implementation) is always tricky and beyond my knowledge, ext2,3 behaves similar to FAT.. XFS, btrfs, ext4 by default is way more fun, I have no idea about UFS/UFS+ in MacOSX. – qdot – 2011-11-16T12:52:35.430
@scottishwildcat: As far as I know there are similar options in most Unixen. Except to get at this option instead of clicking the button to change it you have to wade through oodles and oodles of text files. (Actually, I believe /etc/fstab has all that stuff...) – Billy ONeal – 2011-11-16T18:30:55.200
@nhinkle: Sure, I'd be able to do that. (That chat room seems a ghost town...) – Billy ONeal – 2011-11-16T18:35:27.087
@qdot: There are always device buffers involved too -- some recent HDDs come with some 64MB of volatile memory that's got to be flushed before the disk has power yanked. That's not something (afaik) that the filesystem deals with. – Billy ONeal – 2011-11-16T18:37:14.463
@BillyONeal: Certainly - however, most of that volatile write cache is write-through (or write-back at the first opportunity of transfer, at worst..). It's also configured by the OS - or assumed as such.. My dmesg contains entries like "[487894.462170] sd 10:0:0:0: [sdh] Assuming drive cache: write through" – qdot – 2011-11-17T12:37:04.690
@qdot: Completely depends on the drive. – Billy ONeal – 2011-11-17T14:26:05.230
@BillyONeal: Do you happen to have a reference for it? I'm quite curious which of the drives have unlimited write-back caching? I'm also unsure if they can exist under USB Mass Storage specification, actually. – qdot – 2011-11-17T16:19:44.527
1@qdot: Most consumer level drives don't do much in the way of write back caching. More enterprisey drives almost always have some form of write-back caching either on the drive or controller. Both types are available as external versions. It's not the USB Mass Storage spec we're talking about here; that same dialog is used for any type of device that can be disconnected; e.g. eSATA, FireWire, IBIS/HSDL, LightPeak, etc. – Billy ONeal – 2011-11-17T17:14:37.233
8
The new data is not instantly saved to USB drive, it's stored in RAM for some time. The primary motivation of that appoarch is to gain some performance on pages repeatedly rewritten in place.
So, by unplugging unsafely you have some risk that recently written pages temporarily kept in RAM will never reach the disk.
This may affect not only data, but metadata too - entire directories may disappear, show garbage etc. It's less likely to happen on NTFS, as NTFS has transaction log for the metadata. So on NTFS you just corrupt user data in files and have metadata changes rolled back if you're unlucky on unsafe unplug.
Mikhail Kupchik
Posted 2011-11-15T21:05:47.173
Reputation: 2 381
4NTFS' transaction log ensures consistency of the filesystem. It does not ensure that everything you write to the volume actually gets written. – Billy ONeal – 2011-11-15T21:18:17.053
1It works the other way too. If data if being copied to the computer form an HDD, then it may have been scheduled for delayed transfer. In that case, the data will appear on the destination drive but if the host is unplugged, the destination data will disappear. – AndrejaKo – 2011-11-15T21:24:47.117
Also, the delay in writing to the drive tries to minimise total writes - lengthening drive lifetime. – DefenestrationDay – 2011-11-16T04:18:27.590
@CapsicumDreams: Perhaps for solid state drives that's true. Conventional magnetic storage doesn't have a write count limit. (Well at least not in the "reasonably small enough that someone might hit the limit" territory) – Billy ONeal – 2011-11-17T21:17:34.880
3
Exactly, this is all this does. It flushes all buffers and ensures nothing can access the USB drive anymore by disconnecting it from the system.
sinni800
Posted 2011-11-15T21:05:47.173
Reputation: 3 048
I also checks to be sure the drive is not in use or being written to. – Moab – 2011-11-15T21:08:39.787
It does NOT affect the drive itself. It can however affect the files on the drive. This is because as stated before, files are in cache at the time, so when you eject the drive it will write the files to the drive, otherwise you may get corrupted files. – Matt – 2011-11-17T08:44:14.587
@Matt: Drives have caches too. – Billy ONeal – 2011-11-17T22:38:54.900
@BillyONeal i never said it didnt :), when i say drive i mean the actual platters, from volatile memory to nonvolatile memory. – Matt – 2011-11-18T07:32:20.033
@Matt: How on earth is that not affecting the platters? The eject operation flushes buffers onto the platters. – Billy ONeal – 2011-11-18T14:54:45.560