5

We are having major performance issues with Server 2019. Read and write performance is very poor compared to Server 2016. we installed a number of servers on customer sites to replace old servers and all customers have complained about opening files from the server being slow.

We have set up a test server in our office and measured some read and write speeds and found a vast difference between 2019 and 2016.

We installed a 2016 and a 2019 VM on a 2019 host using Hyper V and used Lan Speed test to measure Read\write speeds to the VMs and the Host

The server is a Dell PowerEdge with 8x 15K disks configured RAID 10 using Intel quad port network card.

The 2016 VM the read\write speeds are 600 Mbps write and 780 Mbps read.

The 2019 VM the read\write speeds are 220 Mbps write and 115 Mbps read.

Both VMs are using the same virtual switch, are configured exactly the same, and are on the same virtual disk, the only difference is the operating system which is fully patched on both VMs.

We have now tested this on various specs of servers at customer sites and found the same results everywhere, 2019 is much slower than 2016.

All BIOS and drivers are the latest available from Dell.

This is not isolated to one server, it is happening on a number of servers, some of which replaced old SBS 2011 servers and yet run slower that the old SBS boxes. Has anyone else found this issue?

repiv
  • 51
  • 1
  • 1
  • 2

3 Answers3

7

You may be seeing this due to Receive Segment Coalescing (RSC). In short, RSC is TCP segment aggregation that combines segments in order to process larger segments versus multiple small segments for efficiency. This was introduced as a default configuration change for Windows Server 2019 and has been the source of issues similar to yours in other environments.

  1. You can disable a client's RSC for IPv4 traffic using PowerShell: Disable-NetAdapterRsc -Name $nic -IPv4.

  2. You can check a Hyper-V vSwitch's current RSC status using PowerShell: Get-VMSwitch -Name $vSwitch | Select *RSC*.

  3. You can disable a Hyper-V vSwitch's RSC in full using PowerShell: Set-VMSwitch -Name $vSwitch -EnableSoftwareRsc:$FALSE. This will not impact existing vSwitch connections.

Items #2 and #3 require you to have access to the Hyper-V PowerShell modules and proper Hyper-V management permissions. As you mentioned seeing this across multiple machines on a Hyper-V vSwitch it is likely that item #3 is where you may find relief. I've provided reference material for you here and here.

Cale Vernon
  • 321
  • 1
  • 4
  • Thanks for your answer, disabling RSC was one of the first things we tried. It makes no difference if RSC is enabled or disabled, and surely if RSC was the issue this would affect both 2016 and 2019. the difference is huge as you can see from the figures I posted. I have seen RSC making a difference but not to this extent. – repiv Jul 24 '19 at 08:16
  • Clients connecting to our virtualized file server were exhibiting extremely slow performance when opening files over the network (3D dental image data). Turning RSC off on the host vSwitch brought a 1-3 minute load time down to 10 seconds or less. Specifically, these were Windows 10 clients connecting to a 2019 file server hosted on a 2019 physical server. – Mike Nov 14 '19 at 15:12
0

I can’t believe Windows 2019 is still somewhat broken after all this time, but I encourage you to try adjusting the following settings and retest:

LAN

Set-NetTCPSetting -SettingName "DatacenterCustom" -CongestionProvider DCTCP
Set-NetTCPSetting -SettingName "DatacenterCustom" -CwndRestart True
Set-NetTCPSetting -SettingName "DatacenterCustom" -ForceWS Disabled

Set-NetTCPSetting -SettingName "Datacenter" -CongestionProvider DCTCP
Set-NetTCPSetting -SettingName "Datacenter" -CwndRestart True
Set-NetTCPSetting -SettingName "Datacenter" -ForceWS Disabled

WAN

Set-NetTCPSetting -SettingName "InternetCustom" -CongestionProvider CTCP
Set-NetTCPSetting -SettingName "InternetCustom" -DelayedAckTimeoutMs 50
Set-NetTCPSetting -SettingName "InternetCustom" -ForceWS Disabled

Set-NetTCPSetting -SettingName "Internet" -CongestionProvider CTCP
Set-NetTCPSetting -SettingName "Internet" -DelayedAckTimeoutMs 50
Set-NetTCPSetting -SettingName "Internet" -ForceWS Disabled

This reconfigures 2019 to behave a little more like 2016.

Bink
  • 183
  • 5
0

Was going to comment, but don't yet have the reputation.

I was seeing this issue on a new Dell T440 server running Server 2019 as the host OS with Hyper-V role, and a test VM (Win10) I spun up.

From my wired laptop - Lenovo P52S - running Win11 I was seeing avg 500Mbs write, but between 90 and 220Mbs read with LAN Speed Test Lite to the test VM. Always using a 20Mb file.

To the host itself, using LAN Speed Test Lite I would see avg 500Mbs write and 500Mbs+ read, so nothing wrong with my laptop (or so I thought)

I disabled RSC on the virtual switch but it made no improvement for me, still poor read performance between laptop and VM, even after rebooting HOST, VM and laptop. I reenabled RSC.

This morning I dug out an old MSI Win10 laptop and gave that a try. LAN Speed Test Lite returned 500Mbs+ write, 700Mbs+ read on first try. Wow! So it was my machine? Ran a few more tests and was seeing as much as 1600Mbs on the read. I then ran twenty more consecutive tests and noticed that on 4 of the 20 tests, read speeds returned lower (about half as low) as the write speed.

Then I (re)disabled RSC on the virtual switch and ran another 20 tests. All 20 returned consistent read > write results, averaging 700-800 on the read, 450-600 on the write.

Reenabled RSC on the virtual switch and 5 of the twenty tests this time saw slow read speed. For me, I'd rather have consistent results rather than a read as high at 1600Mbs+ but as low as 220Mbs so off RSC will stay.

Back on my P52s laptop, still the same poor results but this looks to be Intel NIC related, as when using a USB->Gigabit ethernet adapter with Realtek chipset, speed results mirror that seen on the MSI laptop. I mention this as OP also so no benefit with RSC disabled, as I did initially.

Cheers

Mike

McMuckle
  • 1