Windows 10 IPSec VPN not respecting configured parameters (notably: encryption method)

Question

I am currently trying to establish a VPN connection from my Windows 10 Enterprise 1909 to a remote VPN gateway, using the built-in Windows VPN / IPSec client. Since the UI does not provide all options I need, I have created and fine-tuned the VPN connection with Powershell (using an account with Administrator rights):

Set-VpnConnection -Name Test -AllUserConnection -ServerName other.vpn.gateway -TunnelType Ikev2 -AuthenticationMethod MachineCertificate -EncryptionLevel Required
Set-VpnConnectionIPsecConfiguration -ConnectionName Test -AuthenticationTransformConstants GCMAES256 -CipherTransformConstants GCMAES256 -EncryptionMethod GCMAES256 -IntegrityCheckMethod SHA384 -PfsGroup PFS2048 -DHGroup Group14 -AllUserConnection

I've been very surprised that Windows obviously offers GCMAES256 where it matters; please note that I haven't typed the encryption method etc. manually, but got them from the proposals Powershell ISE offers after having typed an argument.

However, when I dial that VPN connection, Windows claims that there is a "Policy match error". A quick investigation on the remote gateway showed that Windows sends wrong IKEv2 proposals:

We (responder) are not behind a NAT. NAT-T is already enabled
+IKE-SA:
  IKE-Proposal-1  (4 transforms)
    ENCR : AES-CBC-256
    PRF  : PRF-HMAC-SHA1
    INTEG: HMAC-SHA1
    DH   : 14
  IKE-Proposal-2  (4 transforms)
    ENCR : AES-CBC-256
    PRF  : PRF-HMAC-SHA-256
    INTEG: HMAC-SHA-256
    DH   : 14
  IKE-Proposal-3  (4 transforms)
    ENCR : AES-CBC-256
    PRF  : PRF-HMAC-SHA-384
    INTEG: HMAC-SHA-384
    DH   : 14
-Could not match any proposal. See VPN-Debug trace for more information

So Windows sends three proposals, and all of them are wrong. For example, in my Powershell commands, I have explicitly told it to use GCMAES256 for encryption, but obviously, Windows insists on and proposes AES256-CBC.

Could somebody please explain whether this is a bug in Windows or whether I am doing something wrong. Do I have to enable GCMAES256 in Windows somehow?

Furthermore, why does Windows propose SHA1 and SHA-256 (proposals 1 and 2 in the above code) while I have explicitly told it to use SHA-384?

[ Side note: Of course, I could enable AES256-CBC on the remote gateway, but I am considering this only a last resort. I really would like to know what is going on at the Windows side. ]

Answer 1

I have thought of deleting this question because I have found the answer myself, and the problem was not due to a bug in Windows. However, I have learnt that I am not the only one who had this problem, so I leave the question and the answer here for reference. Having said this:

The Windows IPSec stack does still not offer strong authentication, cipher, encryption, integrity and exchange algorithms by default; instead, you have to explicitly enable strong VPN IPSec cryptography. This is true even for the newest versions of Windows 10 (at the time of writing this), and it is a shame that Windows does not offer appropriate options in the GUI where VPN IPSec connections are normally created.

When researching how to enable strong cryptography in Windows, you will inevitably come across at least the following solutions:

1. Create and fine-tune VPN IPSec connections via Powerhell, using commands like those shown in my question.

2. If it does not exist yet, create the following DWORD in the registry:
   HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Rasman\Parameters\NegotiateDH2048_AES256
   Then set it to one of the following values:
   0: Disable AES-256-CBC and MODP-2048
   1: Enable AES-256-CBC and MODP-2048
   2: Enforce AES-256-CBC and MODP-2048

So we have a quick way to enable that strong encryption everybody is after by just editing one registry value, which most of us are used to anyway, right? And of course, we want to be sure that the strong encryption is actually used, so we set that value to 2 to prevent weak proposals from being chosen, or to prevent ourselves from connecting to sites which only offer weak cryptographic algorithms.

A few days later, we realize that this is not enough, because we now want to connect to a site which uses another DH group, for example. So we bite the bullet and learn how to configure VPN IPSec connections via Powershell.

Next, we wonder why Windows does not respect the cryptographic algorithms we have given for a certain VPN IPSec connection via Powershell, and why it sends wrong proposals, and we write questions at this site.

The reason for Windows disrespecting our Powershell VPN IPSec configuration is trivial:

The description for the value 2 in the above registry DWORD must be understood literally. Enforce means, well, enforce, which in turn means that the cryptographic parameters for any IPSec VPN connection will be overridden and that MODP-2048 and AES256-CBC will be used instead.

It does not mean that Windows insists on AES256-CBC and MODP-2048 just if something weaker is configured or offered. It really means that this setting under any circumstances will override other settings given by other configuration mechanisms.

Since I have deleted that registry value, Windows uses and offers the correct cryptographic configuration (i.e. the one given via Powershell) when establishing a VPN IPSec connection. It did not need to be restarted for this change in behavior.

Please note that I did not try what would happen if I set that registry value to 1; I just deleted it because it is not of any benefit as long as you create your VPN IPSec connections via Powershell, which I will always do in the future.

Likewise, I did not research whether it is possible to enable strong cryptography with VPN IPSec connections by group policies or other mechanisms not mentioned above - please leave a comment if you can tell anything about that.