14

Problem: Windows Server 2008 R2 will only support the following ssl cipher suites when using certain certificates on the server:

TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

This prevents XP clients from connecting to the server since the XP Cryptographic API doesn't support any AES ciphers by default.
As a result, the following errors appear in the server logs when attempting to connect using internet explorer or remote desktop. (since they use microsoft's CAPI)

Schannel Error 36874 "An TLS 1.0 connection was recieved from a remote client application, but dodne of the cipher suites supported by the client are supported by the server. The SSL connection request has failed."
Schannel Error 36888 "The following fatal alert was generated: 40. The internal error state is 1204"

Aaron
  • 2,968
  • 1
  • 22
  • 36
Gary
  • 301
  • 1
  • 2
  • 7

4 Answers4

14

If the certificate being used on the server was generated using the Legacy Key option in the certificate request form, the private key for that certificate will be stored in Microsoft's legacy Cryptographic API framework. When the web server tries to process requests using its new, Cryptographic Next Generation (CNG) framework, it appears that something related to the RSA private key stored in the legacy framework is unavailable to the new framework. As a result, the use of the RSA cipher suites is severely limited.

Solution:
Generate the certificate request using the CNG Key template in the custom certificate request wizard.

MMC | Local Computer Certificate Manager | Personal Certificates Folder | (right click) | All Tasks -> Advanced Operations | Create Custom Request | "Proceed without enrollment policy" | select "(no template) CNG key" | proceed to complete the certificate request according to your needs.

Verifying that the key is in the right place:
http://msdn.microsoft.com/en-us/library/bb204778(VS.85).aspx
http://www.jensign.com/KeyPal/index.html

Tools for verifying correct cipher-suites:
http://pentestit.com/2010/05/16/ssltls-audit-audit-web-servers-ssl-ciphers/
https://www.ssllabs.com/

SSL cipher-suite settings:
http://support.microsoft.com/kb/245030
http://blogs.technet.com/b/steriley/archive/2007/11/06/changing-the-ssl-cipher-order-in-internet-explorer-7-on-windows-vista.aspx

This took us a week to figure out. I hope this saves someone the same trouble.

Gary
  • 301
  • 1
  • 2
  • 7
  • Anyone know how to do this - or otherwise solve the issue - for self-signed certificates? – MGOwen Jun 17 '13 at 07:14
  • Thank you very much Gerry for your work and sharing your results! We opened a Microsoft Support Case and Microsoft itself was not able to find out why certain clients could not connect. We have had the Issue exactly as you described. But according to http://technet.microsoft.com/en-us/library/ff625722(v=ws.10).aspx Microsoft itself recommends using the Legacy Key Template to archive the best compatibility! Great Work! –  Jan 29 '14 at 08:57
3

Got this exact same issue myself and this post saved me a ton of time so thanks all!

Gary's solution is spot on but I managed to solve the issue simply by converting the PFX to PEM and then back to PFX again using openssl. The new PFX imported the certificate in IIS just the same with the difference that I can see the missing ciphers.

This is how:

openssl pkcs12 -in mycert.pfx -out mycert.cer -nodes

Then split the cer file in three, the key, the certificate and the intermediate certificate[s]

openssl pkcs12 -export -out mycert-new.pfx -inkey mycert.key \
-in mycert.crt -certfile mycert-intermediate.crt

Then if you import the new .pfx file into IIS it will use all the ciphers you expect to see.

So no need to reissue the certificate.

Angel Abad Cerdeira
  • 171
  • 3
  • This worked for me in a situation similar, but soft of opposite to, the original question: the AD FS role on Windows Server 2012 R2 *requires* you to use the Legacy API for the cert request -- but then it only shows the 2 AES ciphers shown above! Exporting, repackaging the key with OpenSSL, and re-importing solves it the same -- the other protocols suddenly start working. Thanks, @gelilloadbad! – ewall Mar 27 '15 at 14:56
0

Thanks, your post did help me, although my problem was not exactly the same.

However, the cause was a WINHTTP SERVER API configuration mistake on my part. My IP changed, and when I put a new server certificate into the machine "MY", I ignored the ALREADY_EXISTS return code for HttpSetServiceConfiguration.

So I used a previous server TLS certificate which had the wrong common name and did not match my new server name.

If you don't do HttpDeleteServiceConfiguration() or the command line "netsh http delete sslcert 0.0.0.0:8443" correctly you will get a nasty error with these symptoms:

1) In TLS, your Client Hello is immediately met with a TCP packet from your server with Ack and Reset flag bits set. (A handshake failure alert, I think?)

2) The event viewer gets "The following fatal alert was generated: 40. The internal error state is 107."

"An SSL 3.0 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported by the server. The SSL connection request has failed."

So before you go chasing a cipher suite mismatch, ensure that you really are using the server certificate you want to use with:

         netsh http show sslcert

and check the hash values !

AndrewDover
  • 9
  • 1
  • This answer is very unclear and doesn't make a lot of sense. You should aim to directly answer "Why does Window's SSL Cipher-Suite get restricted under certain SSL certificates?" and follow up with an explanation of the Schannel error. – Bernie White Nov 05 '12 at 08:56
0

This could be the KeySpec = 2 -- AT_SIGNATURE issue

certutil -verifystore -v My "Thumbprint of cert" to verify KeySpec value. If its 2 the you will need to export the certificate as a PFX file and then run certutil -importpfx AT_KEYEXCHANGE to fix it.

Michael
  • 11
  • That was the issue in my case as well. In fact, this answer is the only one which actually attempts to point to the cause. The answer would, however, benefit from an explanation why is AT_SIGNATURE not sufficient for non-ECDHE cipher suites - because for such suites RSA is used not only for authentication (signature), but also for key exchange. – Jakub Berezanski Dec 04 '17 at 12:33