Max total files in a directory in FreeBSD 6 ufs?

Question

Is there an upper limit to the total number of files stored in a single subdirectory in FreeBSD 6?

The filesystem is ufs

score 4 · Accepted Answer · answered Aug 13 '09 at 06:53

This was discussed on the freebsd-questions list in November 2008. To quote from Erik Trulsson:

Each i-node on the disk contains a field telling how many hard-links point to that inode. This field is a (signed) 16-bit value, meaning the maximum number of hardlinks allowed is 32767. Each subdirectory created contains a hardlink ('..') to its parent, thus limiting the number of subdirectories to a single directory to less than 32767.

Note that this does not limit the number of files you can have in a single directory, since normal files do not contain hardlinks to the parent directory, but there are of course limits to the total number of files and directories you can have on a single filesystem based on how many inodes were created when the filesystem was first created.

(Full message, start of thread)

These are theoretical limits; as discussed above, you will start to run into performance problems well before you hit any limits.

performance issues may not necessarily be seen until too late - on relatively small partitions I've had many cases where some app will generate many millions of tiny files or link within a hierarchical directory structure that starved that partition of inodes well before space or performance were even warranting investigation. — DisabledLeopard, Aug 13 '09 at 06:59

score 1 · Answer 2 · answered Aug 12 '09 at 20:21

I did some things with freebsd 6.x that involved large numbers of files (50,000+) and don't remember any specific limitations. A quick check on a 7.2 system shows the limit to be well over 100,000. The process is still running as of now, when it fails I'll let you know what the hard limit is for 7.2, which is likely similar to 6.x.

That said, you see a huge performance hit over about 30,000 directory entries when creating new files / directories. At that point people start creating files named HashOfName/name instead of just name so you make lookups easier.

score 1 · Answer 3 · answered Aug 12 '09 at 20:30

1

I would expect that number to vary based on the type of filesystem involved as well.

answered Aug 12 '09 at 20:30

retracile

1,260
7
10

How can I tell which filesystem is in use? – Ian Aug 12 '09 at 20:50
Freebsd installs typically use ffs for their filesystems unless you're doing something exotic or mounting an alien filesystem. The exception is zfs, which is not likely to be used on 6.x. Unless stated otherwise, I would assume ffs/ufs2. – chris Aug 12 '09 at 21:25
Is there a way to tell specifically? – Ian Aug 12 '09 at 21:27
1

type "mount" and you'll see the filesystem type and the mount options. – chris Aug 12 '09 at 21:58
Added ufs to question. – Ian Aug 13 '09 at 06:25

score 1 · Answer 4 · answered Aug 17 '09 at 12:35

Your question is already answered, so just little performance tip: If you have a lot of small files you should increase vfs.ufs.dirhash_maxmem, default 2MB is too small for thousands of files.

I have such line in my /etc/sysctl.conf

vfs.ufs.dirhash_maxmem=67108864

You can read about dirhash here (UFS improvements @ BSDCON) and here (Wiki)

score 1 · Answer 5 · answered Jan 01 '10 at 05:54

The total inodes are the limits of total files you can put in a directory. The total inodes are created when you format your hard drive. You can make more inode by using small size of segments. See man newfs for detail.

The vfs.ufs.dirhash_maxmem is for the memory used to hold the directory names in a directory. This only affects the perfomance, nothing limitations. If you have more memory, make it bigger, otherwise do not bother it.

Max total files in a directory in FreeBSD 6 ufs?

5 Answers5