0

I am having the same issue described in this question: IIS 7 Still Serving old SSL Certificate

I have replaced an old SSL certificate with a new one and completely removed the old certificate from the server.

The poster above solved his issue when he realised that there was an ISA server between his web server and the Internet so he cleared the old certificate from the ISA cache and the problem was solved.

I have seen the same solution elsewhere: http://forums.iis.net/t/1182296.aspx

Unfortunately I am having trouble proving to my ISP that the issue is not on my server and that they are caching the old cert somewhere in their infrastructure.

I am 99.999% certain that the certificate installation and bindings are correct on IIS:

  • I have configured 2 bindings for the site with different alternate ports 9443 and 8443.
  • The old certificate was previously bound to the site on port 9443, both bindings now use the new certificate
  • I have configured the etc/hosts file to point the bound domain to 127.0.0.1 so when I open a browser to the domain the requests are all local
  • If I visit the 2 URLs from another machine (i.e. via the Web) I get served the old cert on port 9443 and the new cert on port 8443
  • If I visit the 2 URLs from a browser on the web server I get served the new cert on both ports

I am 99.999% certain that the certificate is not cached on the client side:

  • I have hit the site from multiple browsers on both PC and mobile devices
  • I have hit the site over multiple connections, including my corporate LAN, home DSL and a mobile

My next step is to do something like a trace route of the TLS session establishment to figure out where the old certificate is being served from so I can prove to my ISP that this is something they need to solve.

Is this possible, what tools are available for this?

Community
  • 1
Adrian Hope-Bailie
  • 255
  • 4
  • 8
  • 1
    Try `openssl s_client -connect server:port`, then feed the certificate to `openssl x509 -text -noout`. – MadHatter Jul 10 '14 at 10:38
  • For external clients: does the hostname resolve to actual ip-address of your IIS server or something else. – HBruijn Jul 10 '14 at 10:38
  • @MadHatter: This just gives be a pretty view of the certificate I am being served. It just proves that it's the wrong one :). Is there a way to trace that connect session verbosely to see the IP of the machine that served the cert? – Adrian Hope-Bailie Jul 10 '14 at 10:56
  • @HBruijn DNS is al working. The host name resolves correctly. Note that the same host with different ports has different behaviour. – Adrian Hope-Bailie Jul 10 '14 at 10:57
  • I thought you wanted to prove that your machines were serving the right one? You asked for help "*proving to my ISP that the issue is not on my server*", and that lets you do that. – MadHatter Jul 10 '14 at 10:57
  • @MadHatter Yes, I see what you were doing there thanks. Unfortunately the ISP still won't accept that the certificate is being cached and served by something other than my server. I need to to somehow trace the SSL handshake to see where the cert IS being served from – Adrian Hope-Bailie Jul 10 '14 at 11:15
  • There is no way that I know of to do what you want. As far as a client is concerned, the SSL handshake is with the IP address that it thinks it's talking to. Since you haven't told us any of the domain names or IP addresses involved, it's hard to add anything more. But you might get some mileage out of using `tcpdump` on the server to show that an SSL handshake that produces the wrong certificate is not actually coming all the way through to your server. – MadHatter Jul 10 '14 at 11:25
  • @MadHatter I don't think it is the handshake (my web server is doing that, it's the only thing with the private key) I think it's just some cache of the public cert. https://nedbank.gyrusonline.com:9443 and https://nedbank.gyrusonline.com:8443 should return a certificate for *.gyrusonline.com. You'll note that 9443 returns a cert for *.stanchiononlonline.com. If I hit these URLs from the server itself (with local DNS entry mapped to 127.0.0.1) I get the correct cert on both ports. – Adrian Hope-Bailie Jul 10 '14 at 11:52
  • 1
    @AdrianHope-Bailie I do not think that your ISP is caching anything in that regards, TBH. They usually do not do this... especially SSL traffic, and escpecially-especially on non-standard ports....! Also, how would they? They don't HAVE your certificate's private key, so it's not even possible... – MichelZ Jul 10 '14 at 11:53
  • And what do you get from the server when you do `echo ""|openssl s_client -connect 127.0.0.1:9443 |openssl x509 -text -noout|&grep depth=0`? The problem with using browsers to check certificates is that browsers do things *other* than build SSL connections, such as requesting content which can include redirection instructions, and following them if it does. It's much better to use an SSL tool to check SSL. – MadHatter Jul 10 '14 at 11:56

1 Answers1

3

I do not think that your ISP is caching anything in this regards, here are the reasons:

  • To serve YOUR certificate with YOUR thumbprint, they need to have access to YOUR private key
  • ISP's usually do not cache SSL traffic
  • ISP's escpecially do not cache SSL traffic on non-standard ports

So this seems to be an issue with your own server. Try rebooting it, try tracerouting it, try the suggestions from the comments like @MadHatter's: echo ""|openssl s_client -connect 127.0.0.1:9443 |openssl x509 -text -noout|&grep depth=0

Also, if you use SNI or stuff like that, make sure that the Default non-SNI Website in IIS has the correct certificate bound, as this is served to at least non-SNI clients, and I have seen problems when they did not match where there shouldn't have been problems.

MichelZ
  • 11,008
  • 4
  • 30
  • 58
  • I agree, check on the server itself to start with. And note that `s_client` does not semd SNI by default, only if you specify `-servername whatever`. But from where I am (Verizon US) nedbank.gyrusonline.com resolves via afrihost-win-1.*.stanchionpayments.com to 197.96.40.118, and :9443 gives me the *.stanchionpayments.com cert without or with SNI. One warning: *on windows* depending on your build, `echo ""|openssl s_client` may require you to hit a key even though input appears redirected. – dave_thompson_085 Jul 10 '14 at 12:19
  • The issue was with Windows' clever HTTP kernel thingy. This answer helped me solve it. https://stackoverflow.com/questions/12866117/the-incorrect-localhost-certificate-is-being-served-by-iis My server had two bindings on port 9443, one for 0.0.0.0:9443 and one for 197.96.40.118:9443. – Adrian Hope-Bailie Jul 10 '14 at 12:23
  • >netsh http show sslcert Shows two bindings on port 9443, one for 0.0.0.0:9443 and one for 197.96.40.118:9443 with different certs. I don't know how those get in there or why the latter was not removed after the config in IIS was changed and IIS restarted. >netsh http delete sslcert ipport=197.96.40.118:9443 Removed the binding for the old cert. My error in debugging was mapping the domain to 127.0.0.1 in the etc/hosts file so local requests used the 0.0.0.0:9443 binding while external requests used the other. – Adrian Hope-Bailie Jul 10 '14 at 12:34