1

All right, so I have work with our networking engineers and it just appears that nobody is able to figure this issue out and so I'm all out of options as I have attempted to Google research this issue to death with no avail. So with that being said, here is the issue and all of the troubleshooting that has been performed. (NOTE: All communications are being done through a website connection utilizing https on Port 443. As well, all of these connections are coming in through an external effacing URL and then going into a Citrix load balancer which is redirecting the connection to the appropriate server where the website service is hosted. The server is running Windows Server 2012 R2 along with IIs 8.5. Yes, all bindings are set. As well, I did make sur ego explicitly enable TLS 1.0, 1.1, and 1.2 through the registry to make sure everything was working correctly.)

I have a connection from another company's website where until recently we have been having no issues at all. The change that happened recently was the fact that we turned off TLS 1.0 and TLS 1.1 and only have TLS 1.2 enabled. As soon as we did this, the connection broke. Now you would think the issue was because the other company was using a weaker TLS connection to connect to us but that has been proven incorrect. So the first step I took was I had my networking team turn both of those TLS versions back on and while running a Wireshark Trace I was able to capture the successful connection to our server. What's weird is that the connection was talking and communicating the whole entire time over TLS 1.2. So this was the first part that confused me.

So the next step that we took was to disable TLS 1.1 and leave TLS 1.0 and 1.2 enabled. After running another Wireshark Trace we confirmed that the connection was still able to be established and that the communications were all still being talked over TLS 1.2.

The next step that we performed was we turned off TLS 1.0 and turned it back on TLS 1.1 as we knew just having TLS 1.2 enabled broke the connection originally. After making these changes and starting another Wireshark Trace, the connection failed.

Once I had all of these logs I went back through each one of them to take a look at what the differences were between all three of them. After lots of digging through them myself, I believe I may have found the issue but I am needing help in confirming it. The one thing I noticed that never happened during the failed connections was there was never a handshake ever attempted at the server level. However, when there were successful connections and the handshakes were all performed as expected on the server level. So that led me to believing that the issue was with the handshake and that it was being terminated at the load balancer.

I took this information back to my network engineering team and talked to them and they are not sure why turning on TLS 1.0 and 1.2 and the optional 1.1 would allow connection when it shows through the Wireshark logs that everything was being communicated over TLS 1.2. So that led me to looking through some of the Cipher Suites that were being used by the vendor and the Cipher Suites that were approved for the connections in the load balancer to my server.

After digging around through those I found out that the the cipher Suite that was agreed upon each time was not one of the approve Cipher Suites that was in the load balancers listings. So that is honestly the largest confusing part about this whole situation is that how could the connection from this other company be allowed through when TLS 1.0 is on but yet all the Cipher Suites that they are offering that they support are not in our approved listings. However, when just TLS 1.2 is turned on then the connection is never made to the server because it appears that it is being terminated at the load balancer.

So with that being said, could anyone explain why turning TLS 1.0 on and yet the Cipher Suites are not in the load balancers approved listing wpuld work but having it turned off would not? As well, what would be the best steps to perform regarding trying to get the connection allowed through. Should we add just one of the highest-rated Cipher Suites that the other company supports into our approved list and see what happens then? Or are there other things that we should look at 2 troubleshoot further and try to find a resolution?

Thanks to anyone and everyone's assistance and advice.

Ryan Wakefield
  • 11
  • 1
  • 1
    I’m confused. Are you terminating TLS at the Citrix LoadBalancer? And are you turning off TLSv1 at the Citrix LoadBalancer or your Windows Server (or both)? Can you give your front end and back end TLS settings? Or better. yet the results from https://www.ssllabs.com/ssltest/ for your server. Also what version and type (VPX, MPX, SDX) of Netscaler are you you running? – Barry Pollard Apr 14 '18 at 18:45
  • Barry, the connection is getting terminated at the load balancer. We turned TLS 1.0 and 1.1 off at the load balancer, not the server. TLS 1.0 and up are still enabled on the server for internal communications only. I did that website and TLS 1.0 and up are all enabled. I don't know the version, but I think (since I dont manage the LB) is a VPX. Front end (external facing) just had 1.2 enabled at the LB. Back end has 1.0 and up enabled. – Ryan Wakefield Apr 14 '18 at 18:47
  • Still need to see that report to help or to know the ciphers in use now (and ideally also what was still there after disabling TLSv1.0). Also what cipher was the client using that you thought as disabled? – Barry Pollard Apr 14 '18 at 19:04
  • For security reasons, I cannot give the specific ciphers that we are using and such. This is just for safety on us. As for cipher I thought the client was using was found in the wireshark logs. I found the cipher that was agreed upon when there were successful connections. However, the cipher agreed upon wasn't in the approved cipher list on the load balancer. So somehow when we turn TLS 1.0 on the LB, it bypasses the approved cipher list or allows additional ones that we were aware of. A side question, can you have certain ciphers enabled on TLS 1.0 only? – Ryan Wakefield Apr 14 '18 at 19:09
  • Then for security reasons I can’t help. If it’s a public website then anyone can figure these out using the tool above so not sure why this is a security issue but hey ho. TLSv1.1 and TLSv1.2 added ciphers by didn’t take anyway (though the upcoming TLSv1.3 is removing all older ones when it is used), so no, you can’t generally have TLSv1.0 only ciphers. – Barry Pollard Apr 14 '18 at 19:15
  • Thank you for your help. As for the website, it isn't a public facing website. It is a private site connection between the other company and us. So the ability for your to access the site and see wouldn't be able to happen as we have a filter on the firewall protecting the URL for only specific IP addresses to be allowed through. Thanks though, we will go ahead and continue troubleshooting on our end in hopes of figuring it out. – Ryan Wakefield Apr 14 '18 at 19:18
  • Np. If it’s not publicly available the. I would suggest you run the testssl tool (available at https://testssl.sh) to give a similar report to ssllabs (which only works on public sites). Run it before and after disabling TLSv1.0 to show which versions of TLS and ciphers are enabled. Perhaps it’s not what you think. – Barry Pollard Apr 14 '18 at 19:22
  • Alright. I will do that test and see what I find out. Thanks again for the auggestions. – Ryan Wakefield Apr 14 '18 at 19:25
  • Was this ever resolved? We have no termination at load balancer and when the server is in TLS 1.0 and 1.2 only mode we fail handshake as well due to unexplained reasons. I have two questions, a) how to list available ciphers for a tls protocol, B) any tool that can help establish a tls 1.0 connection and provide easy to read tls information for debugging? – Anirudh Goel Dec 13 '18 at 06:45

0 Answers0