12

How can lsof report more open files than what ulimit says is the limit?

prod_web3(i-ca0b05aa):~$ sudo lsof | wc -l
4399
prod_web3(i-ca0b05aa):~$ ulimit -n
1024
Adam Nelson
  • 1,557
  • 3
  • 13
  • 12

4 Answers4

14

That's a common mistake to compare results of raw lsof call with supposed limit.

For the global limit (/proc/sys/fs/file-max) you should have a look at /proc/sys/fs/file-nr -> the first value indicates what is used and the last value is the limit

The OpenFile limit is for each process, but it is defined on a user, see command ulimit -Hn for user limits and see /etc/security/limits.conf for definitions. Generally applied with "app user" eg:"tomcat": set limit to 65000 to user tomcat that will apply on java process it runs.

If you want to check limit applied on a process, get its PID and then do: cat /proc/${PID}/limits If you want to check how many files are opened by a process, get its PID and then od: ls -1 /proc/${PID}/fd | wc -l (note: for ls it's 'minus one', not to confond with 'minus el')

If you want to know details with lsof but only for those file handlers that count for the limit, have a try with thoses commands:

  • lsof -p ${PID} | grep -P "^(\w+\s+){3}\d+\D+"
  • lsof -p ${PID} -d '^cwd,^err,^ltx,^mem,^mmap,^pd,^rtd,^txt' -a

Remark: the 'files' are files / pipe / TCP connections / etc.

Note that sometimes you'll probably need to be root or to use sudo to obtain correct result for the commands. Without privilege sometimes you don't have error, just less results.

Finally, if you want to know what 'files' on your filesystem are accessed by a process, have a look at: lsof -p ${PID} | grep / | awk '{print $9}' | sort | uniq

Have fun!

bbodenmiller
  • 117
  • 1
  • 5
Ronan Kerdudou
  • 141
  • 1
  • 4
10

From the ulimit builtins man page

The ulimit builtin provides control over the resources available to the shell 
and to processes started by it on systems that allow such control.

Your lsof command lists all of the open files for all processes for all users on the system. You are not comparing like with like.

user9517
  • 114,104
  • 20
  • 206
  • 289
5

Although this is old, I wanted to ask the same question... the answer is not satisfactory in my case as this is what happens:

$ sudo su tomcat -c "ulimit -n"
1024
$ lsof -u tomcat
3967

I'm not entirely sure why this happened. I guess that open files from sub process are not accounted for in the parent one.

estani
  • 2,011
  • 2
  • 17
  • 12
  • 1
    The limit set by `ulimit` is per process and not per user. If you want to check the number of open file handles for a given process, use `lsof -p $pid` – Læti Mar 28 '14 at 15:48
  • @Tonin I can't remember now exactly, but I'm pretty sure there was only one (main) process. Could it be that the limit applies also to subprocesses (same ppid)? Or perhaps there is some valve configuration that starts new processes instead of sub ones in tomcat... – estani Apr 01 '14 at 16:07
  • The limit value is inherited by child processes. But enforcement of the limit is done per process: the parent can have 1024 open files and the child can have 1024 open files too. I'm not sure about the number of processes tomcat has when running, it might depends on installation/configuration. – Læti Apr 01 '14 at 16:19
3

A bash(1) builtin, ulimit defines the maximum number of open files per process.

This is not a system-wide setting in any way.

adaptr
  • 16,479
  • 21
  • 33