7

On an older Linux box I have, there is an NFS mount of a Netapp. Once there are 100k files in a single directory on this box, files can no longer be written. Anyone know what might be causing this? I have been told symbolic links can still be written.

I will add more specific information as I look it up. Please, skip the 'use a better a file / folder structure and/or database', I know ...

Kyle Brandt
  • 82,107
  • 71
  • 302
  • 444

4 Answers4

7

There are two limits that you may have hit.

  • inodes (unlikely)

    You've consumed all of the available inodes for that volume. You can confirm this with the commands df -i and maxfiles.

  • maxdirsize (more likely)

    ONTAP imposes a limit on size of directories. Size being a product of metadata/hardlinks, not file content. This limit defaults (assuming >6.5) to 1% of your system RAM. It does so to ensure that large directories don't impact system performance. Because linear directory scans require directory data structures to be loaded into memory. Quick overview:

If it's neither of these issues then I'd recommend checking and posting your ONTAP logs.

Community
  • 1
Dan Carley
  • 25,189
  • 5
  • 52
  • 70
  • Oh thanks, looks my thought on walf.maxdirsize was on the right track maybe, but outdated :-) – Kyle Brandt Oct 19 '09 at 18:45
  • Although that would put the max at 20MB, which is a pretty big directory... – Kyle Brandt Oct 19 '09 at 18:54
  • Nope, its not the dirfilesize, its definitely the number of files... – Kyle Brandt Oct 19 '09 at 19:37
  • 20Mb should give you around 640k files NFS-only files. But I've found that the calculations don't always work out so favourably. Nevertheless, do check `/etc/messages`, because the logs are often helpfully indicative. – Dan Carley Oct 19 '09 at 20:13
  • It's not unreasonable to have run out of inodes... within 6 months of moderate usage in an academic research environment I had a system that ran out of the default number of inodes. People can chew up inodes real quick when they do things like store copies of their imap folders in their home directories. – aculich Dec 15 '11 at 07:55
6

The system is hitting the link limit for a directory that is a fixed limitation of the NetApp ONTAP filesystem. From this knowledge base article:

Starting with Data ONTAP 6.5, the maximum number of subdirectories a single directory may have is 99998 (100K). Data ONTAP 6.4 and earlier versions were restricted to 65534 (64K) subdirectories. This number may not be changed. To understand the reason for this limit, see the section below on hard links and subdirectory implementation.

Kyle Brandt
  • 82,107
  • 71
  • 302
  • 444
  • 1
    This answer is not true anymore, the limits can be changed, see new answers http://serverfault.com/a/591995/10361 – sorin Apr 29 '14 at 12:51
1

http://www.unixunderground.com/blog/?cat=11

A bit dated, but might be helpful. I believe the number of files is based on the number of disks/nodes.

Mojeska
  • 166
  • 4
0

On newer versions of NetApp (8.1+) you can overwrite the default limits per volume using this command:

vol options vol_name maxdirsize 150000

To see current settings run 

vol options vol_name

There is no way to change these from the GUI client.

sorin
  • 7,668
  • 24
  • 75
  • 100