Limit of 21842 files in a directory?

16

Copying files using Fedora 16 from EXT2 to a NTFS 2TB drive. Discovered that when recursively copying directories of many files, copying stops at file 21842 in each directory... then cp -r moves on to the next directory. No error is given.

Searching the web, found someone else reporting this problem about FAT32 to no avail.

I can create well over 21842 files on the drive on the same system... just can't seem to copy over that number.

This is the command being used:

cp -r /media/BAKKER_UPPER/many_files_here/* /media/NEW_NTFS_HOME/ 

What's going on here? How do I get my files over onto the NTFS drive?


I am not hitting the upper file limit on number of files in an NTFS directory (unlimited). Nor am I am hitting the max number of files for the drive.. (~4billion). I also have lots of free blocks left on the drive.

jedierikb

Posted 2012-07-15T15:30:38.043

Reputation: 509

How do you know that that is how many files are getting copied? – soandos – 2012-07-15T16:44:27.187

2What's the exact command you're using? – ott-- – 2012-07-15T16:57:31.210

2I have a hunch that you are running into the classic "argument list too long" issue. Are you trying to copy files based on a filename pattern? Or are you copying all files in the directory tree? – iglvzx – 2012-07-15T16:59:43.723

cp -r /media/BAKKER_UPPER/many_files_here/* /media/NEW_NTFS_HOME/ – jedierikb – 2012-07-16T02:46:15.540

5Do it without the wildcard (it's not needed anyways). Shell expansion is going to turn that wildcard into a giant string of source files that will be limited to the number of bytes the shell can accept as an argument, and thus the number of files you can copy. – Garrett – 2012-07-16T05:42:36.020

2@Garrett Why don't you post that as an answer as it is quite likely to be the right answer. – Mokubai – 2012-07-25T07:41:47.313

@Garrett indeed it is / was – jedierikb – 2012-07-25T12:26:11.287

1Done :) (upvotes appreciated!) – Garrett – 2012-07-26T01:29:49.850

Answers

10

Do it without the wildcard (it's not needed anyways). Shell expansion is going to turn that wildcard into a giant string of source files that will be limited to the number of bytes the shell can accept as an argument, and thus the number of files you can copy.

So your new command would be:

cp -r /media/BAKKER_UPPER/many_files_here/ /media/NEW_NTFS_HOME/ 

Garrett

Posted 2012-07-15T15:30:38.043

Reputation: 4 039