How can I have my process detect if antivirus injected a module or DLL to it?

Question

I am writing an installer process (.exe). My installer deploys different components. It will add registry entries, copy files, copy files over the network, remote execute, remote PowerShell, local Powershell, etc.

Sometimes, antivirus DLLs are injected into my process and harm my installer function flow causing it to fail (blocking it, killing it, etc).

Questions

1. Are there any best practices or guidelines out there to detect this?
2. Are there any best practices or guidelines out there to eliminate this problem? (FYI, we signed the DLLs/EXE.)

An example approach:

1. Start the installer, when its up and running, Use procmon or ListDLLs, get the loaded DLLs list and check if any of the DLL belongs to known antivirus or EDR suite. If so, disable it.

I'm looking for a highly reliable way to figure out if antivirus or EDR is interfering with my process. All my .exe and DLLs are signed.

Answer 1

How can I have my process detect if antivirus injected a module or DLL to it?

This may be difficult for your process to do.

In the example you provided

An example approach: Start the installer, when its up and running, Use procmon or ListDLLs, get the loaded DLLs list and check if any of the DLL belongs to known antivirus or EDR suite. If so, disable it.

a different process (procmon) is used to analyze your process. In fact, this is not just a separate/different process, but it is also a privileged process. Any similar watchdog process will likely have to be privileged because it will need to access other processes' memory (e.g., your process's memory).

If you start to modify your process to perform introspection on itself (e.g., to look at it's own loaded DLLs and compare them to known good DLLs), this behavior itself will likely seem fishy to AV, and that might make your problems worse.

If your installer is using a lot of cryptographic operations the AV might also think this is fishy.

In general, you could suggest testing the installer on various platforms and seeing what AV interfere with your installer. They in your documentation you can warn that these AV cause problems. You could also try to contact the AV vendor and see if they can whitelist your application, but I'm not sure how successful you will be unless you are making a very widely used software.

Finally, if you really are trying to "disable" AV injecting DLLs, you will likely need your own kernel driver to be installed. The AV itself likely has a kernel driver that "listens" for any new process and immediately injects its own DLLs. To counteract this will likely require a kernel driver. And, even then, it might not work.

Answer 2

AFAIK, this is a well known issue with anti-virus softwares, and little if any can be done.

A decent antivirus not only has signatures for well known malwares but also use heuristics to detect weird activity from unknow programs. In order not to prevent usage of well known valid softwares, they also have a white list for programs that should no be inspected with heuristics.

The problem is that an installer is a piece of code that does a lot of suspect activity: it writes new files in system folders, eventually modifies some, may download executable files from network, etc. That alone is enough for anti-virus software to see it as suspect if they do not know it as a well knows program.

Because of that, and before the generalization of white lists, a good deal of installer programs used to ask the user to switch their antivirus off before installing their application.

Even now, you are left with only 2 ways:

• declare your application and its installer to all major anti-virus vendors and ask them to white-list it. If you are from a well known company, it is really the way to go. If you are a free-lance developper your demand could be ignored...
• provide your users ways to make sure that they have a genuine installer from you (md5 or better sha hashes, secure distributions, etc.) and ask them to switch their anti-virus off for the installation. Yes it is the (not so) good old way, but this one is bullet proof...

Answer 3

The general task of this kind has no solution.

The operating system, the high permissions processes or services can do practically whatever they want. The various malware "vendors" trying evasive measures are in a constant arms-race with antiviruses.

On the other hand, legitimate antimalware software would not "inject" anything. What it usually does is to deny you a permission for some action (writting a file, changing a registry value) or, in some cases, silently fail or revert your action. Or kill your process for good.

Signing your installer package and probably your executable files may be a good thing because the user, the admin or you (given access) can recognize files that are corrupted or tampered with. It may be as well used to whitelist your software in the operating system or in the antimalware software.

Other than this, it looks like you are trying to write a malware. Good to see you are that much naive. :)