Does Windows still copy each file directly to disk, unlike Unix?

Unix was originally designed to perform file system operations in memory, and to update the disk blocks only every 30 seconds during sync. This resulted in a much faster user experience than MSDOS many years ago that accessed the disk blocks directly for each file system operation. But today using Windows 7 I still see that copying lots of small files takes much longer than copying one big file of the same size.

For example: a Visual Studio C# Asp web project Publish directory contains 2000 files, total 150 MB, zipped (7Zip) 90 MB.

Copy the entire directory to the same disk: 30 seconds
Zip: 5 seconds
Copy one zipped file: 2 seconds
Unzip: 65 seconds. (On another machine: 20 seconds)

Computationally, unzipping should go faster than zipping. So the unzipping might be dominated by the file system performance.

Disk fragmentation might be a factor, and the reason another similar machine might be faster in handling many file system operations. But if disk fragmentation is a factor here, it might indicate that the user has to wait for each file system operation to finish before the next one starts, i.e. like old DOS.

Roland

Posted 2014-06-25T12:58:56.227

Reputation: 299

How Windows copies a file has not changed. But I also don't believe Windows uses the disk entirely to copy a file. – Ramhound – 2014-06-25T13:03:42.080

Trying to find info to answer my own question I found that Windows does apply disk caching: [http://msdn.microsoft.com/en-us/library/windows/desktop/aa364218%28v=vs.85%29.aspx] . But the MSDN pages are full of remarks on how to avoid caching. And we are of course not talking about 256 KB objects here. – Roland – 2014-06-25T13:25:19.677

Answers

From the info I found myself (see above) and from contributors above, I conclude that modern Windows can do disk buffering/cache, but that this is often disabled. Reasons include that NTFS has no journalling. Windows performance might be of secondary importance than profits: quality versus time to market. With Unix/Linux, which are less commercial, quality/performance is much more important.

Coincidence: a coworker is currently zipping a few GB of files to try to copy faster, and is staring at a Windows Archive dialog box saying "10 seconds to go" for the last half hour, with a slowly growing zip file (not frozen). But that might also be a faulty disk... :-)

Roland

Posted 2014-06-25T12:58:56.227

Reputation: 299

With exception of Mac osx which gives performance at way higher price – STEEL – 2014-06-26T02:52:27.910

The behavior (uncached access) you describe hasn't been in use since Windows 3.11. However, Windows flushes its cache more aggressively that Unix. It's down to their respective roots. Unix machines were assumed to be built with reliable hardware and power. Windows PC's were cheap boxes, which crashed far more often. That wasn't just an OS problem, UPS'es are far less common on PC's and cheap IDE disks were less reliable on powerloss than expensive server SCSI disks. The ancient FAT format didn't help either - no journalling so massive corruption possibilities. NTFS is much better. What did you use for the test?

MSalters

Posted 2014-06-25T12:58:56.227

Reputation: 7 587

My machine has an NTFS formatted disk. – Roland – 2014-06-25T13:21:46.597

I believe that the reason was not just hardware. Unix has always been a multi-user multi-process system needing lots of disk operations. DOS started with almost no disk at all. There was not much point to write to a DOS-1 floppy that could only hold 64 file names or something similar. All subsequent DOS/Win versions tried to be backwards compatible, so today we have still drive letters. Still curious about caching today. – Roland – 2014-06-25T13:35:17.693

Contradicting myself: DOS stood for DISK Operating System. – Roland – 2014-06-25T13:37:07.713

1Downvote issued for a statement that cannot be backed up by facts "Windows PC's were cheap boxes,which crashed far more often" is simply false and cannot be backed up. Personal Computers for nearly 2 decades didn't even use Windows, once Microsoft was in the picture, the reliability of the personal computer market was low for a varity of reasons none had anything to do directly with Windows or Microsoft. – Ramhound – 2014-06-25T13:42:30.150

@Ramhound: read the answer again. I'm contrasting Windows and Unix. That is why I can write "crash more often" - it's a comparison. I didn't say Windows PC "crash often". – MSalters – 2014-06-25T13:53:31.107

@Ramhound, MSalters : don't you both believe the origin of DOS direct write to disk has mostly to do with the low ambition target of DOS? A Personal computer would not need a multi-user/process system, or high performance file operations? And PC/DOS low reliability had to do with Bill Gates inventing how to make money with software, i.e. time_to_market ? – Roland – 2014-06-25T14:09:58.880

@Roland: I'm old enough to remember the pre-PC days. The Apple II was probably less reliable than the original IBM PC. Remember that IBM was a solid computer designer back then, quite possibly the most experienced company. They did take shortcuts in the PC design, sure, but it wasn't outrageously unreliable for the time. – MSalters – 2014-06-25T14:14:51.343

@Roland - No. I don't believe that. I had a PC I purchase in 1992 and it still works to this day. Bill Gates had no control over the quality of the Personal Computer being sold, Microsoft only sold software, the PC in question was an IBM Compatible PC. Any relability issues came from the fact IBM went from the sole source market for PCs to multi-source everyone man and women for yourself market of clones. The quality problem with clones also existed with Apple when they allowed such a thing. – Ramhound – 2014-06-25T14:16:51.783

The PC runs/ran MS-DOS then Win 3.1 for Workgroups. – Ramhound – 2014-06-25T14:21:14.403

The thing with reliability is that a single sample says nothing. In reality, what makes quality expensive is the need for consistency. All million power supplies produced should produce 12.00 Volts +/- 0.01 volt. The cheap supplies produce 12.00 Volts +/- 1.00 volt. Still, you can be the lucky buyer who gets a cheap power supply doing precisely what it should do. – MSalters – 2014-06-25T14:24:42.187

Seems like we can agree that at the time of origin of MS-DOS the hardware was not the problem. Seems like nobody disagrees that the original MS-DOS ambition target was low. About hardware: curious thing that most Linux machines run on the same hardware as Windows. – Roland – 2014-06-25T14:36:34.897