We are really stumped on this. During an upgrade to a newer Apache environment running on CentOS 6.6 we encountered weird problems with TLS connections from Windows 8 machines. The first hint at the error was users of IE10 complaining as well as .NET applications running on Windows 8 calling our web services.
We installed a modern.ie VM of Windows 8 running IE 10 and Fiddler 4. We noticed that in Fiddler 4 the tunnel command would periodically show the output:
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 14:59:42.679
Connection: close
fiddler.network.https> HTTPS handshake to hostname.com failed. System.Security.Authentication.AuthenticationException A call to SSPI failed, see inner exception. < The logon attempt failed
When we look in the Windows Event viewer on the machine we see Windows Event 36887 and the message.
A fatal alert was received from the remote endpoint. The TLS protocol defined fatal alert code is 49.
So we then moved over to the web server. The server is running:
httpd.x86_64 2.2.15-39.el6.centos
httpd-tools.x86_64 2.2.15-39.el6.centos
mod_ssl.x86_64 1:2.2.15-39.el6.centos
openssl.x86_64 1.0.1e-30.el6_6.7 @updates
We setup a DEBUG log to gather information about the traffic. What we would typically see in the output is this:
[Tue Mar 24 17:38:08 2015] [debug] ssl_engine_io.c(1935): OpenSSL: I/O error, 5 bytes expected to read on BIO#31854001770 [mem: 3187c0110c3]
[Tue Mar 24 17:38:08 2015] [debug] ssl_engine_kernel.c(1882): OpenSSL: Exit: error in SSLv3 read finished A
[Tue Mar 24 17:38:08 2015] [debug] ssl_engine_kernel.c(1882): OpenSSL: Exit: error in SSLv3 read finished A
[Tue Mar 24 17:38:08 2015] [info] [client 123.125.71.101] (70007)The timeout specified has expired: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
Now other browsers like IE11 and Chrome on Windows 8 will also generate the same errors in the server logs, but it's like they have retry logic built in and will power through. We tried everything we could think of:
- Changing SSLCipherSuite to a variety of strings.
- Disable TLS v1.1 and v1.2
- Serve up local content from the apache instead of content from the backend application server reached via mod_proxy.
- Turn KeepAlive Off
When we scan the server with Qualys SSL Lab is shows compatibility with everything except IE6/XP and Java 6 and scores an A.
It's almost like Windows is having issues reusing an SSL connection or something. Not seeing any errors out of FireFox on Windows 8 because it uses it's own library instead of SChannel.
Any help is greatly appreciated!
Thanks, Jonathan
Update 1 - 03/25/2015
We decided that some of the issues might be caused by the backported cipher work done by RedHat and an incompatibility with something in Apache 2.2 or Mod_SSL. Lot of moving parts. We decided to bite the bullet and build out a CentOS 7 on Apache 2.4 since it has updated Mod_SSL and OpenSSL libraries compared to the 6.6 branch. That appears to have fixed all the Windows 8 SChannel errors that we were seeing. I suspect this will eventually come to RedHat's attention and a fix will eventually be issued for the problem.
Update 2 - 03/25/2015 Can't get the IE10 to have the SChannel error with the CentOS 7 server, but some .Net clients were still reporting errors like this:
[Fiddler] The connection to secure.hot.com' failed.
System.Security.SecurityException Failed to negotiate HTTPS connection with server.fiddler.network.https> HTTPS handshake to secure.ultracart.com failed. System.Security.Authentication.AuthenticationException A call to SSPI failed, see inner exception. < The logon attempt failed
Update3 - 03/27/2015 Turns out that a firewall in front of the web server that does protocol inspection did not like something about the TLS packets being sent by Windows 8 and was terminating the traffic.
