6

My terminal server users experience a delay when selecting printers from MS Office applications to be printed to network printers. Everything stalls on:

Finding available printers...

The environment is a 4-server Windows 2008 R2 RDS farm. The printers are configured on a dedicated Windows 2008 R2 print server local to the network.

All of the RDS servers experience the delay, however the issue seems to be isolated to Microsoft Office 2010 applications. Adobe, web browsers, etc. are not impacted.

It’s a 5-7 second enumeration delay under normal use, and up to 35 seconds at the busiest periods of the day RDS server.

Here's a video to show the timeline...

enter image description here

There are 16 network printers in this environment using universal drivers where applicable.

enter image description here

Edit:

I already went through the process described at:
2008 R2 Terminal Server: "Insufficient system resources exist to complete the requested service"

Community
  • 1
ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • If you export the following registry keys to a file, how large are the files? `[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Printers]` and `[HKEY_USERS\.DEFAULT\Printers]` – Greg Askew Sep 08 '15 at 14:23
  • @GregAskew They are 1,977KB and 12,784KB, respectively. – ewwhite Sep 13 '15 at 12:52
  • Does the latency happen on a clean user profile? – yagmoth555 Sep 13 '15 at 22:56
  • @yagmoth555 Yes, a new profile sees the same thing. – ewwhite Sep 13 '15 at 22:57
  • I seen you dont have much driver type. Did you try with a user with only HP driver vs one with only konica printqueue ? – yagmoth555 Sep 13 '15 at 23:02
  • 1
    I ask as I remember the old KB from the chat, so I know Office read the status of each printer. In the printqueue if its the case you can remove 'birectional' support. That remove the printer software monitor and such. In fact, if the printer monitor is buggy, removing the support will help the query. You cant see after 'low ink', 'no paper' and such warning, but that remove a lot of network talk, and remove the printer monitor from the vendor – yagmoth555 Sep 13 '15 at 23:16
  • Is it possible that some communication be blocked between your RDP server and printers? Some drivers like to do `snmp` calls, some even `http`, etc. Which could cause slow if the driver is depending on that to get the current status of the printer. – ETL Sep 14 '15 at 00:52
  • Long-shot, but this issue might be because of a default printer that was set, and then changed on the printer server. If you try changing the default printer on one of the affected stations to Microsoft XPS, does the issue persist? – Reaces Sep 14 '15 at 12:24
  • @Reaces Changing to XPS and Adobe PDF each had no impact on the printer enumeration time. They remained at 6 seconds. – ewwhite Sep 14 '15 at 12:39
  • Did you try using a user profile with less printers? My guess Office is enumerating each printer and getting more specialized settings (if available trough driver) like margin offsets etc.. Try a profile with only one specific type of printer driver and check if this issue is with all drivers – eKKiM Sep 14 '15 at 13:06
  • @eKKiM No. The environment has 16 printers across three facilities, and users require access to all of them. – ewwhite Sep 14 '15 at 13:08
  • I understand that the users require them all, but just for testing purposes i would create accounts to split them up. I think there is an issue with one of those 4 drivers you use. – eKKiM Sep 14 '15 at 13:15
  • I'd try to launch Process Monitor and then filter the entries which takes more than 4-5 sec to figure out why it takes so long to enumerate printers. – Volodymyr Molodets Sep 18 '15 at 08:32
  • @VolodymyrM. It's the `spoolsv.exe` process. – ewwhite Sep 18 '15 at 09:29
  • @ewwhite: if you view event properties - does it list any print drivers under modules section of the window? – Volodymyr Molodets Sep 18 '15 at 11:01
  • @ewwhite - don't know, maybe this could be helpful: http://blogs.technet.com/b/askperf/archive/2012/02/24/microsoft-fixit-for-printing.aspx – ETL Sep 25 '15 at 17:59
  • @ETL Please make this an answer. – ewwhite Sep 25 '15 at 20:55
  • @ewwhite - i guess that helped? – ETL Sep 25 '15 at 21:08

3 Answers3

4

How many of you have ever dealt with an issue where you just knew that something was wrong with your print spooler but could not quite put a finger on it? Maybe print jobs were slow, certain users could print to some printers but not others, or maybe nobody could print at all? - Blake Morrison - Ask Performance Blog - Microsoft Fixit for Printing

The quoted article references two Microsoft FixIt which basically cleans up the Spooler settings and restore it, etc.

Direct link to Print Reset Full Mode - http://go.microsoft.com/?linkid=9829711 Direct Link to Print Reset Lite Mode - http://go.microsoft.com/?linkid=9829710

There are two modes - full and lite. The lite has less things it does. The blog post details what the FixIt does behind the scene.

ETL
  • 6,443
  • 1
  • 26
  • 47
3

In case it helped, will write an answer with what we talked.

Please check Performance issues due to Inactive Terminal Server Ports

There are several issues that have been associated with a high number of inactive Terminal Server ports. Delayed logon times to RDP sessions, failure of printers to redirect, and slow server performance due to registry bloat from all the ports. These inactive TS ports accumulate because the Remote Desktop Services Device Redirector service creates a new port every time an RDP session is established, but the ports are not always recycled. Every RDP session can possibly create a new port, and every ended session means a new inactive port. Performance degradation is known to occur when 250 or more TS ports exist in the registry. Increasingly large numbers of redirected devices will exacerbate performance delays.

The resolution:

Long logon time when you establish an RD session to a Windows Server 2008 R2-based RD Session Host server if Printer Redirection is enabled

and run that FixIT to clean the registry.

yagmoth555
  • 16,300
  • 4
  • 26
  • 48
  • I had over 8,000 entries in the registry. I've removed them, but cannot reboot the server to make the hotfix active just yet. – ewwhite Sep 08 '15 at 16:52
  • 3
    I went through and applied this fix to the terminal servers. However, it did not actually resolve the issue with the printer delays. It probably fixed _other_ performance problems and was much-needed, but the original issue remains. – ewwhite Sep 13 '15 at 14:18
2

This is an recurring issue when using horribly written print drivers. In this situation there was two suspects (which both are guilty); HP Universal Print Drivers and Konica Minolta Universal Driver.

For some reason both these drivers refuse to run in anything other than CSR mode (Client Side Rendering). On a terminal server this can get disasterous, as they populate the same keys over and over and over inside HKEY_USERS\.DEFAULT\Printers, just with a different GUID each time. Combine that with users who have every printer on the planet mapped to their userprofile, and you get a shitstorm of printer installation every time they log off.

In this specific situation the terminal servers had millions of entries inside the registry hive.

The steps to "solve" this has been:

  • Install MS hotfix 2778831 if you are running 2008 R2, and MS Hotfix 2871131 if you are running 2008 R2 SP1 or 2012 R2.
  • Keep the list of drivers as minimal as possible. Remove the driver packages that you don't need
  • Update the drivers (this stopped the HP driver from spamming the registry, Konica driver still sucks)
  • Set drivers to shared isolation mode, and change the print processor to winprint in hopes of stopping CSR from activating
  • Clean out all the junk from HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider
  • Stop Windows from deleting and re-creating printer connections every time a user logs off (or disconnects) by setting RemovePrintersAtLogoff=dword:00000000 in the key HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider. Remember to restart the spooler.
  • Clean print software from HKEY_USERS\.DEFAULT\Software and printer connections from HKEY_USERS\.DEFAULT\Printers
  • Boot a Windows PE image and compress the registry
  • If using HP UPD in a managed environment - install the group policy templates from HP Managed Print Administration and disable all the extra "features", like the popups about toner remaining and super deals on new toners and such. It slows down the spooler, as it has to trigger a new process to start every time you so much as look at the printer inside Windows.
  • Do not install full printer application packages on a terminal server. Just use normal drivers on a shared print server, without any kind of "easy" discovery methods or dynamic print targets.

I suspect that one just has to do the tasks above at intervals. Maybe it could be scripted.

Do you wonder, after reading all this, if you too have the same problem? Go to %SystemRoot%\System32\config and check the size of the file DEFAULT. If it's anything larger then a few hundred MB's then it's time to put on your detective hat.

pauska
  • 19,532
  • 4
  • 55
  • 75