java.io.IOException: Too many open files on tomcat server

Question

I have web application which is based on JAVA (GWT Framework). I hosted my web application on Redhat Linux Server 6.0 using Tomcat 8.0.9 in front of apache http server connected using AJP connector and JDK 8. Whenever 20+ users access my website tomcat goes failed and that i checked the logs file i got this execption

19-Oct-2015 12:38:15.350 SEVERE [ajp-nio-8009-Acceptor-0] org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
    at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:688)
    at java.lang.Thread.run(Thread.java:745)

Web Application uses :

Database : couchdb 1.5 (Running on ec-2 machine using 4gb RAM)

Search : elasticsearch 1.0 (Running on ec-2 machine using 4gb RAM)

Tomcat : Tomcat 8.0 in front of apache server connected using ajp connector(Running on ec-2 machine using 8gb RAM)

I searched alot & find out the solution to change the values of fs.file-max = 200000 and soft file and hard file limit to 65535 but still this error occur and tomcat stop responding. Then i need to restart the tomcat ec-2 machine then only application start running properly.

I could not understand why this problem occurs very frequently?

My clients get irritating because of this problem. Please help me out of this problem

See http://serverfault.com/questions/122679/how-do-ulimit-n-and-proc-sys-fs-file-max-differ — Federico Sierra, Oct 19 '15 at 16:43
i already do that my ulimit is 4096 which is less than the file-max then whats the problem ? — Kushal Jain, Oct 19 '15 at 16:47

score 4 · Answer 1 · answered Oct 19 '15 at 17:18

4

Try increase open file limit as below (in /etc/security/limits.conf):

*         hard    nofile      65535
*         soft    nofile      200000

and restart your services.

You can check limits of a running process:

cat /proc/xxxx/limits

where xxxx is the pid of process

answered Oct 19 '15 at 17:18

Federico Sierra

3,499
1
18
24

which process i need to check ? tomcat process or couchdb or elasticsearch ? – Kushal Jain Oct 20 '15 at 03:29
@KushalJain Try with lsof command (eg: `lsof -p | wc -l`) that list all open files used, and then check the limit for all process – Federico Sierra Oct 20 '15 at 03:53
pid is of tomcat process id ? – Kushal Jain Oct 20 '15 at 03:55

score 0 · Answer 2 · answered Aug 01 '18 at 06:07

You need to check on the tomcat process as below:-

$ ps aux | grep apache:

I was also experiencing the same but above check was fine. I had to back-up some files in the tomcat logs directory. I then forcefully killed the tomcat process as below:-

kill -9 <tomcat-process-id>

and then restarted tomcat which now worked correctly

java.io.IOException: Too many open files on tomcat server

2 Answers2