Why does moving large folders take a lot of time?

I have a large folder with >100k files. I moved it into my archive folder and its taking forever to move. Why is that? I know on XP it takes less than one second but not on Windows 7. I am sure its a permission thing, is there a way I can disable it and make it faster?

I am moving the folder into another in the same drive/partition. In XP. As far as I know it just moves the folder file from one place to another. In Windows 7, it seems like its touching something in every file when I move it.

What can I do to fix this? Drop permission properties?

user3109

Posted 2010-03-26T21:51:40.393

Reputation:

3I don't know why you would be so sure that it is a permission thing. Take another look at other variables that may have changed (network lag, copying to a USB drive, etc.) What other info should we know about what you are moving, from where are you moving the files, and where to are you moving the files? – Chris Dwyer – 2010-03-26T23:01:44.143

Wait for SP1??? – Lawrence Dol – 2010-03-27T07:39:47.783

I'm wondering this too, some tings I intended to try but not having had the time yet: Is it filesystem specific; i.e. does it matter whether its FAT32 or NTFS? Disable indexing for the filesystem (in case the time goes into updating the search indexes). – user12889 – 2011-02-04T02:40:05.283

@user12889: I am not sure what i did but ATM on my NTFS drives it moves instantaneously. I dont know if its because i disabled "Remote Differential Compression" (http://maximumpcguides.com/windows-vista/disable-remote-differential-compression/) or if it was something fixed in one of the windows updates.

– None – 2011-02-04T04:56:47.830

Answers

Right now folders move in an instant. I remember in the past turning off "Remote Differential Compression" http://maximumpcguides.com/windows-vista/disable-remote-differential-compression/ but ATM it is checked off so i believe it was something fixed in one of windows updates.

user3109

Posted 2010-03-26T21:51:40.393

Reputation:

It happens when I use Windows Explorer to move (or to cut and paste).

The only way around this that I know of is to use something other than Explorer to move the directory. For example from Windows' cmd.exe, using move a b moves even large directories instantaneously. The same goes for Cygwin's mv command.

Rainer Blome

Posted 2010-03-26T21:51:40.393

Reputation: 406

1How does this answer the question in the title? – soandos – 2012-09-23T05:09:42.877

1The answer implied in my post is "Because Explorer was used". The original question "Why does moving large folders take a lot of time?" lacks info on how the move was done. Because of that, if I answered "it happens when you use Explorer". I might be off the mark. I do know that Explorer is the cause on my system, and I share this with you. Surprising as it may seem, before I posted, no one explicitly named Explorer as the culprit. Yes, I'd like to know why Explorer is so slow. As long as there is no known fix, I give a workaround using on-board means, answering "What can I do to fix this?". – Rainer Blome – 2012-09-24T09:11:53.673

When a folder appears to move instantly, it's because the operating system has been able to update the file allocation table* without moving the actual file data.

It's slower to do this with small files, as for each one a change has to be made in the table. If the files are really small, this may even take a similar amount of time to actually moving their data.

I don't know in exactly what circumstances the file data has to be moved when it's on the same partition as its destination but I would assume there's nothing you could do to avoid it. As other commentators and answers have hinted at, copying to a different drive (different disk, different partition, to a USB stick, to a drive over the network, etc) would of course mean you had to copy the full data, so things will be slower. You will be affected a lot more by the bandwidth of what you are copying to.

(*Think of file data as books in a library, and the file allocation table as a set of index cards showing you which section the books are in)

RJFalconer

Posted 2010-03-26T21:51:40.393

Reputation: 9 791

i looked at the FAT32 format before. I think instead of moving the folderid its updating permissions on every file because the new parent folder may have new permissions. I dont need any permissions here so and so if i could turn it off and move fast i'd do it. – None – 2010-03-27T20:22:26.147

It appears that whenever Windows Explorer tries to move (or copy) a folder, it takes an extra validation step. If this takes over a couple seconds, you can see the status message "Discovering items". This appears to be taking inventory of all of the files in the folder; in the same way that actually opening that folder in Explorer does.

This step may be needed in some cases, but it should not be needed for most "move this folder from here to there" operations. To further speculate -- it seems plausible that doing this validation solved some bug in a better way for some cases, and Microsoft never thought it important to improve performance for the odd-cases of having "more than hundreds" of files in a folder. Such problems might include: when merging folders; if the destination folder is on different media/partition; maybe to check if new file paths will be too long for NTFS. IMO, any/all of these could have been checked in advance, to avoid doing this extra indexing operation, but I know that edge-case performance is often ignored.

Abacus

Posted 2010-03-26T21:51:40.393

Reputation: 321

I haven't used XP in ages, but just be aware that moving large quantities of small files is much slower than moving small quantities of large files, at least in Windows 7.

Also, moving files between the same drive is much faster than moving files between different drives, sometimes instantaneous.

unrelativity

Posted 2010-03-26T21:51:40.393

Reputation: 685

i edit the question. – None – 2010-03-27T01:23:31.017

This doesn't necessarily answer WHY it happens..

http://www.codesector.com/teracopy.php

I've been using TeraCopy as of late and noticed it does seem to copy a bit faster.

NoCarrier

Posted 2010-03-26T21:51:40.393

Reputation: 3 481

a bit faster or moves instantaneously like it did on XP? – None – 2011-02-04T01:55:23.200

A bit faster (if at all). It still does not do it anywhere near instant. – user12889 – 2011-02-04T02:37:05.003

Try using the command prompt command

copy source destination.

I used this and 20k files copied immediately.

pat capozzi

Posted 2010-03-26T21:51:40.393

Reputation: 19

Just an FYI - SuperUser is for answers and nothing else. I edited your answer to reflect that. Good answer, though! – Jon – 2014-05-07T04:21:41.783