I agree with the comments to your question that you probably can't prevent the RDP detection, at least not with reasonable effort.
However, you could try to use alternative software to provide the remote access, for example VNC (there are a lot free VNC servers and clients out there). If you are lucky, the software won't detect that it runs in a VNC session.
As a last resort, you might consider to move the system where the software should be installed into a VM and let the VM pipe the screen contents to you.
For example, I have a lot of VMs running under Linux / KVM. KVM is able to pipe out the screen contents in VNC format, so I can use a common VNC client to view (and even operate) the remote system.
Please note that this is completely different from installing VNC into the remote O/S where it could be detected. Instead, it is more like taking the video cable between the PC and the monitor and add taps to extract the video signal at the hardware level to make it visible to you without touching the O/S. In most cases, this cannot be detected by normal software.
DISCLAIMER
You might violate licensing rules in following my advice above. There are too many forms of licenses out there, and I am not a lawyer. But please be assured that you definitely could violate a license by executing the respective software in a remote desktop environment even though the license text does not say a single word about remote desktop environment. It depends completely on the license text. If unsure, ask the legal department of your company.
What does the license say? Is it actually legal to use it remotely? – Eugen Rieck – 2019-01-15T09:18:20.767
It doesn't specify anything regarding remote desktop use as far as i know. The error message is along the lines "Error executing from Remote Desktop". – Anima – 2019-01-15T09:19:52.903
Does it use a Smartcard device? – Eugen Rieck – 2019-01-15T09:20:17.180
No, it's just a "normal" installation of the program. And then executing it by starting the .exe. – Anima – 2019-01-15T09:22:30.330
There are many ways to detect an RDP connection, many of which can't be disabled (e.g. DirectX resolution change) – Eugen Rieck – 2019-01-15T09:26:10.567
1If we knew what the software was we might be able to offer a reason why it does not work on a remote desktop... – Mokubai – 2019-01-15T09:37:03.763
A lot of software is still written to detect and block installation inside an RDP session. I suspect if you could get it installed, it would actually -run- even in an RDP session. – Debra – 2019-01-15T22:37:48.420