7

We have this Apache on Windows Server 2012:

Server version: Apache/2.4.9 (Win32) Apache Lounge VC10 Server built: Mar 17 2014 10:48:43

Most of the time, it works perfectly, but from time to time (randomly, cca 3-4 times per week), it stops serving HTTPS on port 443, but continues to serve HTTP on port 80. When it stops serving HTTPS, the only way to resolve the problem is restarting Apache.

This seems to be the very same problem as is here: Apache stops responding to http requests -- https continues to work except that in our case it is HTTPS that stops working and HTTP is the one that works all the time.

We have enabled trace6 level of verbosity to get some good information on what is going on. However, the logs are empty:

...
[Sat Mar 21 07:51:50.577373 2015] [ssl:debug] [pid 3356:tid 2540] ssl_engine_io.c(999): [client ...:16529] AH02001: Connection closed to child 137 with standard shutdown (server ...:443)
[Sat Mar 21 07:54:21.936742 2015] [ssl:info] [pid 4760:tid 432] AH01914: Configuring server ...:443 for SSL protocol
...

In this log, we can see the last line for the last request that was served OK at 07:51:50 and then there is nothing (our internal monitoring system attempts to connect every minute and so there should be 07:52:50 record, but it is missing). The next line at 07:54:21 is after our internal monitoring system timed out and restarted the Apache service.

Our internal monitoring is in C# and its output was:

System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at ExecuteServicePageCheck(Object stateInfo)

Our httpd-ssl.conf looks like this:

  <VirtualHost _default_:443>
        ServerName ...:443
        ServerAlias www.....com
        DocumentRoot ${US_ROOTF_WWW}/.../www/www
        SSLEngine On
        SSLProtocol All -SSLv2 -SSLv3
        SSLHonorCipherOrder on
        # Prefer PFS, allow TLS, avoid SSL, for IE8 on XP still allow 3DES
        SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+AESGCM EECDH EDH+AESGCM EDH+aRSA HIGH !MEDIUM !LOW !aNULL !eNULL !LOW !RC4 !MD5 !EXP !PSK !SRP !DSS"
        # Prevent CRIME/BREACH compression attacks
        SSLCompression Off
        # Commit to HTTPS only traffic for at least 180 days
        Header add Strict-Transport-Security "max-age=15552000"
        SSLCertificateFile ${US_ROOTF}/core/apache2/server_certs/....crt
        SSLCertificateKeyFile ${US_ROOTF}/core/apache2/server_certs/....key
        SSLCertificateChainFile ${US_ROOTF}/core/apache2/server_certs/....ca-bundle
        CustomLog "logs/.../www/access.ssl.%Y.%m.%d.log" combined
        ErrorLog "logs/.../www/error.ssl.log"
        LogLevel trace6
    </VirtualHost>

Is there anything more we could do to get any information on what is going on there? We have no error message from Apache anywhere, not even on this trace6 level.

Note that we do have more domains served on both HTTP and HTTPS and the problem occurs for all of them on HTTPS. It is like if Apache just closed that port silently.

Community
  • 1
Wapac
  • 652
  • 4
  • 15
  • 2
    When it stops responding, have you checked the output of `netstat` to see if Apache or anything else is listening on 443? Do the Windows event logs show anything useful? – dartonw Mar 22 '15 at 03:12
  • 1
    Good idea with netstat, but unfortunately no. We have first spotted this problem a long time ago, during the development and implemented an internal monitoring system to detect the problem and restart Apache. Today, the whole solution is in production for several months already and we just realized that it need to be fixed, because it starts to be a problem for us to have even these small downtimes. As for Windows event logs - yes, this has been examined and there was nothing related. Currently we are trying to investigate the issue and find a solution via trial/error method tweaking our conf. – Wapac Mar 23 '15 at 10:06
  • This is sounding like a systemic problem with Apache from the volume of similar google results. How do we bring this up with the developers who clearly released a half-baked product? – ajeh Jan 14 '17 at 14:48

2 Answers2

9

After testing various configurations, we found out the problem and the server runs stable for 7 days now without any problems.

This settings in httpd.conf file fixed the problem:

AcceptFilter http none
AcceptFilter https none

Previously, we only had the AcceptFilter for http and the second line was missing.

According to Apache documentation the default values on Windows are:

AcceptFilter http data
AcceptFilter https data

Using none value uses accept() rather than AcceptEx() and will not recycle sockets between connections.

Wapac
  • 652
  • 4
  • 15
  • I read someone else's question regarding the very same problem as I also have had this problem. I have 32 GBs of RAM as does the other person (not in this post) and someone suggested possible memory issues. I doubled my RAM space a few months ago and believe I had these Apache issues about the same time. Do you think there's something to this? By the way, I think your solution has worked so I'm very grateful to you! – nxasdf Dec 21 '15 at 03:29
  • I did not observe anything that would suggest any relation to amount of available memory. However, in our case, there was always enough free memory, so I do not have any data from a low-memory system. – Wapac Feb 25 '16 at 08:06
  • 1
    Looks like the default accept filter has been changed to 'connect' on Windows. "The data AcceptFilter (Windows) For versions 2.4.23 and prior, the Windows data accept filter waited until data had been transmitted and the initial data buffer and network endpoint addresses had been retrieved from the single AcceptEx() invocation. This implementation was subject to a denial of service attack and has been disabled." – frmdstryr Sep 12 '16 at 15:56
-3

Well I think that is a firewall problem you need to install iptable to pass some IP. Enabling filter should solve the problem

petalkenny
  • 1