3

This is a pretty odd situation. I'll try to outline the details and make it as clear as possible.

We had an ERP client on an old Citrix environment that used Windows Server 2008 R2. I moved the client to Windows Server 2016 and now, some users are experiencing an issue where the connection seems to disconnect after about 1-2 hours of being idle (for the most part). I should also mention that the old environment was also on a separate network. So I haven't ruled out GPOs, network configuration, etc.

Here are the errors that the users see when the ERP (SXE) client disconnects from the server:

Errors

Googling those errors brings up articles like this one from Progress (the maker of the DB, IIRC). But the resolution is a very generic explanation. I also checked Event Viewer and the crash does get logged:

Event Viewer

But again, Googling the error brings up generic resolutions. "Check the network" is the gist of it. But that could mean dozens of things.

Here's what I tried so far:

  • Changed local policy for RDP session timeout to never.
  • Changed local policy for RDP keep alive to ping every 1 minute.
  • Added executable to DEP exception
  • Added some users to admin group
  • Checked with network team, they gave the all clear.
  • Checked with the SAs who manage group policy and they nothing looks off. The only thing they thought of way the screen saver setting, but they said it wasn't set.
  • Currently running Wireshark on the server to see if anything odd stands out.

The most perplexing thing is that it doesn't effect everyone. I opened the client in Citrix with both of my accounts and the app stays open all day. While someone right down hall from me gets disconnected 8 times a day.

Sorry for the long winded post. Any ideas on what I can try to resolve this issue?

ernest
  • 107
  • 1
  • 7

2 Answers2

1

It seem an error from the backend. The app versus the database, as the receiver close the connection, but it’s because the published application failed as seen, not because the receiver lost the network.

Edit 1; as talked in comment, to rule out a citrix/network error or an application error, please connect an user to a full remote desktop there to make him work from there. If the application error still come, you will know it's a backend problem (database) or the application is not fully compatible with 2016.

The citrix just connect both endpoint and display the feed, the error you see seem to me a bad network or a bad load on the database server.

I would suggest to collect data on the database server to be sure it can handle the load, make sure the database is optimized (maintenance done, transact log not to big, etc..) and that the disk are optimized, example if in esx (paravirtual controller or lsilogic disk, etc..)

For the fact an user get disconnected 8 times a day versus you just let me think that user do work more into the app, and must trigger data intensive request.

yagmoth555
  • 16,300
  • 4
  • 26
  • 48
  • Do you have any ideas on what I could do to pinpoint the issue between the client and the server? Maybe I didn't this super clear, but prior to the migration of the **client** from the old server to the new, this never happened. The database and host server have not changed. So I'm not inclined to believe that the database or host server itself is having problems. – ernest Jan 18 '19 at 16:49
  • @ernest The citrix reveiver will close if the exe crash, thus you think it’s a network issue BUT you application log clearly show application error – yagmoth555 Jan 18 '19 at 16:50
  • The event viewer showing the crash is pretty normal since the app itself is installed on server. But just because the app crashes, doesn't necessarily mean it's an app problem. If the something on the network caused the connection to close prematurely, the result would be the same. I don't believe it to be an app specific issue due to the fact that it did not have this problem on the old servers and network. – ernest Jan 18 '19 at 16:53
  • @ernest to rule that out, give a user a remote desktop there and watch if while he work if the app crash, but keep in mind if the network crash, citrix reconnect the connection, the app run in the server, and still run even if the user is disconnect – yagmoth555 Jan 18 '19 at 16:56
  • the issue occurs as they'r idle for an hour or so. the RDP session will time out before the app has an issue. But even still, the issue does not happen with everyone. If it was strictly an app issue, I would be getting time outs as well. But I'm not. The app stays open for 12 hours for me. Thanks for the suggestions though. – ernest Jan 18 '19 at 17:19
  • Just FYI, as I suspected, it was a network problem. Increasing the TCP session timeout resolved the issue. – ernest Jan 25 '19 at 04:25
  • @ernest glad you found your issue. The Firewall was where? I didnt seen that detail in your question – yagmoth555 Jan 25 '19 at 04:35
  • I don't have many details on the firewall, as that's a different team that manages it. The firewall was the eventual solution, so it couldn't have been a part of my question. But it's nice having one less thing to worry about. – ernest Jan 25 '19 at 04:43
0

The problem was with the network. The client needed a consistent TCP connection to the server. When the connection was severed, the client didn't know how to reconnect and led to these errors. When investigating the issue with the networking team, we found that TCP connections on the firewall were set to expire every hour (3600 seconds). We've increased the timeout and the issue disappeared.

enter image description here

ernest
  • 107
  • 1
  • 7