RDP to computer from same computer fails. Why?

Background: We have software that (remotely) retrieves information from a Windows desktop. The desktop needs to be locked, but if we do that the simple way, Windows thinks no-one is looking at the screen and stops updating it. To prevent that, we open an RDP session to the computer. This locks the screen as a side-effect, but the original desktop still gets updated.

We have a small program (called 'lockscreen') based on the Remote Desktop Active X Control to open an RDP session. It appears to behave itself just like the Microsoft Terminal Services Client (mstsc.exe) for all intents and purposes of this question. Any behavior described below holds for both lockscreen and mstsc. All results are from a Windows 7 computer.

Running the program on the target machine, when trying to connect to the target machine (localhost), the error "Your computer could not connect to another console session on the remote computer because you already have a console session in progress" appears (errorocode 1800). This happens regardless of whether I use 'localhost', 127.0.0.1, the name, or the IP address of the target computer. It appears to be a check in the Remote Desktop Active X Control that can be bypassed by using the IP address 127.0.0.2 to access the local machine. Using 127.0.0.2 indeeds allows the connection to proceed and a logon attempt to be made. However, then the remote desktop only shows "Access Denied" and an OK button. Pressing the button or waiting a minute causes the session to be disconnected.

I have been unable to find out why this Access Denied error is given. There are no entries in the event log that shed some light on this. It only happens when trying to connect from the target computer itself: connecting from another computer succeeds.

Some experiments I did: When trying to connect from another computer but going through a proxy that runs on the target computer, it succeeds - it does not matter whether the proxy connects to localhost or to 127.0.0.2. When connecting from the target computer, going through the same proxy, it fails, again regardless of whether the proxy connects to localhost or 127.0.0.2. It seems that only the location of the program setring up the connection is relevant.

In all the scenarios described above, FreeRDP manages to connect successfully. Apparently there are no authentication or authorization issues, or any other fundamental issues that would prohibit a connection to be made.

My two questions: Why am I getting access denied when setting up an RDP session from one computer to itself? Is there anything I can do about it?

remote-desktop

Jeroen ter Hofstede

Posted 2019-11-08T14:30:08.903

Reputation: 1

Does the target computer have to a physical one? Because I would move this (legacy) desktop software into a virtual machine. Then it is protected and you can configure it to never lock, no screensaver, ... without any security implications. – Robert – 2019-11-08T16:14:48.660

Target computer can be either VM or physical. What software it runs is up to the customer to decide, and the same holds for the security requirements. We cannot put restrictions on that. – Jeroen ter Hofstede – 2019-11-11T07:42:58.427

So we are not talking about one PC running this set-up but a whole fleet of installations!? If I were you I would really try to get rid of this bad software that requires an unlocked desktop. – Robert – 2019-11-11T08:43:22.203

Answers

This is indeed working as designed. Normally, you cannot initiate an RDP session to the same computer, due to a loopback.

It is possible to patch the terminal services .dll file to allow this, but it is not a nice solution, because with every windows update, it will break, and the patch have to be applied again, which is difficult because you have to search for a patch for the new version of the file.

I would look into the reason why the desktop needs to be locked. Maybe you can adjust that so the machine is not actually locked, or the lock is different.

For example, you can choose to switch to a different user. The first user is still running in the background in a "locked" state, but it may still be in a state that works for you. The other user is in a different state and you can test to lock that user or maybe even log out.

LPChip

Posted 2019-11-08T14:30:08.903

Reputation: 42 190

FreeRDP manages to connect to the same computer without the terminal services .dll file being patched. I guess that it's something in the communication between the server and the microsoft RDP client libraries that triggers this behavior, which gives me slight hopes that it can be circumvented if identified. – Jeroen ter Hofstede – 2019-11-08T14:55:36.820

The question becomes can you use FreeRDP and connect to localhost without a problem? If so, that is of course a suitable workaround – LPChip – 2019-11-08T15:10:59.563

Indeed - unfortunately FreeRDP has other issues. This question actually comes from trying to find an alternative for it :) – Jeroen ter Hofstede – 2019-11-08T15:16:09.667

Have you tried it without RDP as suggested in my answer? Simply make sure a 2nd user is on the computer and switch to that user, so both users are logged in at the same time? – LPChip – 2019-11-08T15:18:50.683

Yes, but that causes Windows to stop updating the screen for the first user, so it is useless for our purpose. RDP keeps the desktop 'alive'. I know of no other way to achieve that. – Jeroen ter Hofstede – 2019-11-08T15:25:54.430

Have you considered running everything inside a VM? that way, you can RDP into it externally, or run the vm in a windowless state (headless). Oracle VirtualBox is very good at this. – LPChip – 2019-11-08T15:26:34.393

Everything already runs inside a VM (at least, in many actual configurations). RDP-ing from the outside is a possible workaround we're looking into, but it has numerous drawbacks in our application's use case. – Jeroen ter Hofstede – 2019-11-08T15:38:17.773

Let us continue this discussion in chat.

– LPChip – 2019-11-08T15:39:33.377

Connecting to Terminal Server service running on localhost (or IP 127.0.0.1), ends in session disconnected or unable to connect. This behavior is explicitly baked into RDP by Microsoft, and the reason for it is lost in time.

Below are some workarounds that bypass that limitation. As you can see, the Microsoft developer that put this in RDP didn't exert himself. The workarounds may still work for the latest Windows 10, but I haven't tested it.

Connect Remote Desktop Connection to 127.0.0.2, which is an alias to 127.0.0.1.
Connect at non-standard port, other than the default (3389).
Run mstsc.exe simulated as another operating system.
This involves copying from the C:\Windows\System32 both mstsc.exe and mstscax.dll to a new folder, right-click the copied mstsc.exe, selecting Properties, Compatibility tab, and tick "Run this program in compatibility mode" for "Windows 98 / Windows Me".

Note that when logging in in Remote Desktop session, do so as a different user, or you may be logging yourself out forcefully.

harrymc

Posted 2019-11-08T14:30:08.903

Reputation: 306 093

Thanks. I tried all these and they don't work. As I wrote, using 127.0.0.2 prevents the error "Your computer could not connect to another console session on the remote computer because you already have a console session in progress" from occurring (the check apparently baked in into the RDP client library), but then it gets me stuck on the "Access Denied" screen after connecting. – Jeroen ter Hofstede – 2019-11-11T07:40:00.893

So this bypasses one problem. Try: (1) mstsc /admin, (2) the workaround in this post.

– harrymc – 2019-11-11T07:48:32.613

Unfortunately, that doesn't work. – Jeroen ter Hofstede – 2019-11-11T10:14:22.910

Probably then the purpose of this limitation on localhost was to avoid a bug in RDP. The limitation can be worked around, as in my answer, but RDP just cannot work across the loopback adapter. It seems like there is no solution except using a third-party product such as FreeRDP, TeamViewer or other. Or of going through the internet (if the router supports it). – harrymc – 2019-11-11T10:36:29.033

Yes, I fear that's the case. – Jeroen ter Hofstede – 2019-11-11T11:35:55.073