SEVERE: Servlet.service() for servlet [CfmServlet] in context with path [/] threw exception

Question

We have been running CF 11 U5 for a while now with no problems at all. We run CF Enterprise on IIS 8.5 on Server 2k12 64bit.

Recently a few of our clients have been complaining that when they upload files via our various methods of file uploads they get errors back. These errors are browser errors telling them connection has been reset, or page cannot be display etc.

This only seems to happen once the file size goes above 300-400kb. We have no restrictions in IIS as 95% of our other clients have no trouble uploading files up to 200MB.

Nothing has changed to any of the code that runs these uploads. We have a mixture of standard pages with form post which CF then uploads the image to some pages that use jquery file upload plugins which post to CF pages that do the upload.

It doesn't seem that anything is ever uploaded and the user gets the error back pretty quickly within 2-5 seconds of starting the upload. The uploaded files are usual images (jpg) or zip files.

This only seems to affect a limited amount of people as well. When every I try to replicate the issue uploading the same files from my side I never see the problem and everything works as intended. A lot of our other clients have no issues uploading either.

Looking at the IIS logs they are getting a 400 error. Failed request logs tell me its a bad request.

When looking at the Coldfusion error log I get the following error when they attempt it:

org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [CfmServlet] in context with path [/] threw exception
java.io.IOException: Socket read failed
  at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:325)
  at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:447)
  at org.apache.coyote.ajp.AjpProcessor.receive(AjpProcessor.java:343)
  at org.apache.coyote.ajp.AbstractAjpProcessor.refillReadBuffer(AbstractAjpProcessor.java:685 )
  at org.apache.coyote.ajp.AbstractAjpProcessor$SocketInputBuffer.doRead(AbstractAjpProcessor. java:1209)
  at org.apache.coyote.Request.doRead(Request.java:430)
  at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:290)
  at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:449)
  at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:315)
  at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:200)
  at com.oreilly.servlet.multipart.BufferedServletInputStream.fill(BufferedServletInputStream. java:64)
  at com.oreilly.servlet.multipart.BufferedServletInputStream.readLine(BufferedServletInputStr eam.java:108)
  at com.oreilly.servlet.multipart.LimitedServletInputStream.readLine(LimitedServletInputStrea m.java:60)
  at com.oreilly.servlet.multipart.PartInputStream.fill(PartInputStream.java:93)
  at com.oreilly.servlet.multipart.PartInputStream.read(PartInputStream.java:191)
  at com.oreilly.servlet.multipart.PartInputStream.read(PartInputStream.java:152)
  at com.oreilly.servlet.multipart.FilePart.write(FilePart.java:252)
  at com.oreilly.servlet.multipart.FilePart.writeTo(FilePart.java:211)
  at coldfusion.filter.FormScope.fillForm(FormScope.java:338)
  at coldfusion.filter.FusionContext.SymTab_initForRequest(FusionContext.java:512)
  at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:33)
  at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
  at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
  at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:151)
  at coldfusion.CfmServlet.service(CfmServlet.java:219)
  at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.j ava:303)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
  at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
  at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.j ava:241)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:422)
  at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:199)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.jav a:607)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.lang.Thread.run(Unknown Source)

The problem affects a few of our clients, but they are experiencing it 100% of the time. So its not an intermittent problem for them. This error appears on every attempt they make. For all our other clients they have no problem at all.

This error is where I am stumped. What could cause this error for a select few people uploading files?

Any ideas?

Since it is only happening with such a small portion of your user base it seems likely an issue on their end don't you think? So what hardware/software is between their users and your server? Seems like something in the connection might be the culprit here. Do you know someone from their "IT" team that may be able to help you troubleshoot? Perhaps they could try connecting to your server while bypassing some of their network appliances/proxies and see if the issue remains. — Miguel-F, Oct 13 '15 at 14:29
When I say clients I mean completely unrelated companies located in different parts of the country. All have different hardware, networks etc. Some of these are pretty small that just have one computer straight out to the net. Nothing complex in terms of networking. Surely a connectivity issue their side wouldn't cause a Coldfusion error server side though? — haxtbh, Oct 13 '15 at 14:33
It may if it is altering the request. In your question you state that IIS reports _Failed request logs tell me its a bad request._. — Miguel-F, Oct 13 '15 at 14:39
My point here is that you say 95% of your clients are working fine. It is only this subset of clients having an issue. While you are getting an error in your logs I would still look into what is different with these failing clients versus your working clients. It could be any number of things that is causing the issue. Could be a connection timeout. Could be a specific file type they are uploading. Could be something in how they create the file they are trying to upload. Files created on a Mac versus a PC. Etc, etc. — Miguel-F, Oct 13 '15 at 14:54
We have been through most of this doing like for like file uploads. Using the files they try we get them to send them to us via ftp and we do the same. We have got them to swap a router in one case, remove any switches etc. They are all on different ISPs. Two have the same fibre product with the same out the box ISP hardware and one works the other doesn't. This code base hasn't changed in a long time and this is only a recent thing. They don't seem to have any issues with other services that allow file uploads. — haxtbh, Oct 13 '15 at 15:06
Alrighty then. Well technically your error _java.io.IOException: Socket read failed_ is a connection issue but the cause of which could be several different things. I found [this one referring to a timeout setting](http://serverfault.com/questions/389834/ioexception-socket-read-failed-using-apache-2-mod-jk-and-tomcat-over-ajp). Other than that I'm afraid I have no more ideas. I still believe this has something to do with those clients though and not so much your server. Something is different on their end or at least different enough to set your code off into an error. Good luck! — Miguel-F, Oct 13 '15 at 16:47
Nope all on the local drive. Same drive as CF is installed on. Timeouts don't seem to be the issue as this can happen within a few seconds of uploading. — haxtbh, Oct 13 '15 at 18:25
I am not sure, why I didn't get a notification, for your comment. java.io.IOException: Socket read failed, comes due to Network errors only. Did you try taking thread dumps, when this occurs? You will get more insight then. — Anit Kumar, Oct 21 '15 at 13:44
Reinstalled CF in the end and the issue went away. None the wiser to what is was unfortunately but nothing else on the server changed, network or connectivity wise. Instantly started working correctly for these users after the re-installation. — haxtbh, Oct 22 '15 at 16:13

SEVERE: Servlet.service() for servlet [CfmServlet] in context with path [/] threw exception

0 Answers0