0

We are experiencing an issue on our classic Azure VM running Windows Server 2012 R2 when accessing www.linked.com.

We have some applications running on IIS and talking with LinkedIn OAuth API thats where we noticed the problem. But simply trying to access www.linkedin.com in IE we get the following:

enter image description here

The following is an error we get in .NET application:

enter image description here

We went through a lot of internet resources related to changing registry keys, applying updates, etc. but no luck. We discovered that accessing the same site on a new type of Azure VM, hosted in the same region, is no problem at all. Same registry settings, same IE security settings but the classic VM still refuses to open the site.

It seems like a component sitting above the classic VM isn't resolving the network requests correctly. This has started only in the past few days as it worked absolutely fine in the past.

Other TLS 1.2 enabled websites like Google or Office365 open up with no problem on either of the VMs.

Any suggestion, ideas - much appreciated.

What was attempted to fix the issue:

  • Followed the instructions in the IE error window.
  • Used IISCrypto to confirm TLS settings.
  • Updated/checked Windows registry settings suggested here and here (as well as other websites suggesting similar things).
  • Mainly tried to find the difference between classic and new Azure type VMs when a network request goes out using WireShark and a few other tools.

Here is Wireshark output of a request being made from a new type of VM (where IE has no issue opening linkedin.com) (screenshot includes ClientHello TLS content).

enter image description here

And here is the same request in the classic VM

enter image description here

Not a 100% sure what that is telling me but maybe will help someone when sharing their insights.

nlv
  • 121
  • 2
  • Can you post the contents of the "Client Hello" for both VMs? That's where the initial TLS protocol offering happens; I'm quite sure the two VMs are offering different protocols and LinkedIn doesn't like the offer from the classic VM. – Massimo Jul 15 '20 at 14:37
  • Screenshot updated, let me know if you need to see content from all available "bullet points" – nlv Jul 15 '20 at 14:48
  • Does this answer your question? [Random TCP RST's on certain websites, what's going on?](https://serverfault.com/questions/641794/random-tcp-rsts-on-certain-websites-whats-going-on) – Michael Hampton Jul 15 '20 at 15:03
  • Based on the Wireshark capture you posted, I think you are suffering the same problem as in that linked question. Its solution should help you. – Michael Hampton Jul 15 '20 at 15:03
  • Nice catch @MichaelHampton, it looks like the new VM is not sending ECN while the old one is (wonder *why*, though). – Massimo Jul 15 '20 at 16:52
  • @NeilVarnas can you post the output of `netsh int tcp show global | find "ECN"` for both VMs? – Massimo Jul 15 '20 at 16:54
  • @Massimo Server 2012 had it on by default, at least in its initial RTM build. – Michael Hampton Jul 15 '20 at 16:58
  • @MichaelHampton Yes I know. What I find strange is that it *still* has it on by default, even in Azure (I deployed several WS2012R2 Azure VMs last month and I checked them just now, it's on). But based on the capture I see here, the "working" VM is not using it... – Massimo Jul 15 '20 at 17:50
  • 1
    Gents, I read the linked question several times to understand what's going on and it did sound promising but after disabling ECN on the classic VM - no dice. See a screenshot [here](https://ibb.co/F3qVDgT) classic VM not working (left), right new type VM working (right) – nlv Jul 15 '20 at 18:00
  • Can you show a fresh wireshark capture now that you've disabled ECN? – Michael Hampton Jul 15 '20 at 18:01
  • Wireshark capture [here](https://ibb.co/xGXvBmj) (classic VM) – nlv Jul 15 '20 at 18:08
  • 1
    After comparing the two Wireshark captures found that the number of cyphers sent are different, [screenshot here](https://ibb.co/6vPTng4) (left not working, right working). – nlv Jul 15 '20 at 19:01
  • 1
    @NeilVarnas That's promising. Even when both parties are using TLS 1.2, if they can't agree on the cypher exchange this will lead to a fault. – Massimo Jul 15 '20 at 22:40
  • Thanks @Massimo and Michael Hampton for all suggestions, see my answer below. – nlv Jul 16 '20 at 06:27

4 Answers4

3

First, follow the instructions it gave you regarding enabling TLS 1.2. LinkedIn's web site now requires TLS 1.2 and accepts no older TLS or SSL version.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
2

After digging some more into two Wireshark captures (one from the working server and one where the issue occurs) I found that a different number of cyphers were being sent on Client Hello stage of the request, see screenshot here (left classic VM, right new VM). I used IISCrypto tool to compare cypher selection between two machines and right enough, matching the selection has fixed the issue. I also noticed that Multi-Protocol Unified Hello was not selected for both Server and Client Protocols (see screenshot below). Whether this was part of the problem or not is down to further testing and investigation.

Not entirely sure how this has happened but one of the colleagues suggested that a certificate change on the LinkedIn side might have caused this issue.

enter image description here

UPDATE 16/07/2020:

After some additional digging by my colleague GW, he found the exact cypher that was missing and was required to make successful calls to LinkedIn using a custom TLS tool to display TLS connection.

TLS tool can be found here and the output of that showing the cypher used in the call to LinkedIn below (Diffie-Hellman Ephemeral Aes256 Sha384 - TLS_DHE_RSA_WITH_AES_256_GCM_SHA384)

TLS tool output for LinkedIn call

nlv
  • 121
  • 2
0

Windows virtual machines (VMs) that you create in Azure by using the classic deployment model can automatically communicate over a private network channel with other VMs in the same cloud service or virtual network. However, computers on the internet or other virtual networks require endpoints to direct the inbound network traffic to a VM.

You will need to check the endpoints and ACLs. Some docs on this are belwo.

https://docs.microsoft.com/en-us/previous-versions/azure/virtual-machines/windows/classic/setup-endpoints

https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-troubleshoot-connectivity-problem-between-vms

Ken W MSFT
  • 594
  • 2
  • 6
0

To my knowledge, outbound network traffic works the same way for Classic VMs and ARM VMs: the traffic is simply NATted, there are no proxies or anything else inspecting it (unless you specifically deploy such a solution).

The error seems to be related to TLS, thus confirming that your VM actually is able to reach LinkedIn at the network level: it just can't successfully negotiate a TLS session with it.

I'd check again the TLS settings in the affected VM, and also double-check if it can successfully access other websites that require TLS 1.2 (such as Office 365 or Azure services).

Also, please note that if you deploy a new Windows Server 2012 R2 VM in Azure (ARM), it will use a more recent OS image which will already include TLS 1.2 support; this might not be the case for an old VM which has been running for a while in Azure Classic.

Massimo
  • 68,714
  • 56
  • 196
  • 319