What kinds of PE file infection can AV software detect?

Question

I wonder what kind of infections AV software is capable of detecting:

• only if the file is signed?
• also if the entry point isn't changed?
• does it also have hashes for non system files?
• what if the signature is simply removed?

Answer 1

Different antivirus software packages have different detection capabilities and mechanisms. Virtually all support what is called a signature-based mechanism, where a particular series of bytes serves as a fingerprint that triggers a positive detection.

It's important to understand this use of the word "signature" is completely different than a digital signature, which is a cryptographic value mathematically proving the origin of a digital document. An AV file of malware detection strings should be more properly called a "virus fingerprint file" or "virus fingerprint blacklist", but the word signature was used long ago and it stuck.

AV software does use actual digital signatures, however, to validate that their update files haven't been tampered with.

The problem with detection strings is they are static. If a malware author realizes his code is detectable, he just changes his code a little bit, which changes the fingerprints, causing it to no longer be on the blacklist.

To help with this, AV software also commonly uses heuristics, which is logic that detects typical virus behavior. Such behavior might be "trying to act like a debugger", or "installing a hook in a different program". Any software that attempts to do these activities might trigger a detection warning. Heuristics are a good addition because they can work on new viruses that have never even been invented yet.

Better than detecting malware, however, are AV tools that use application white-lists. These programs will simply refuse to allow an application to run that doesn't have a valid checksum on file in the whitelist. They are difficult to use, however, because they require a sophisticated setup and constant maintenance as software is added and updated.

Another solution is to configure an operating system to require digital signatures on all code before allowing it to run. This is how Apple's iOS locks out anything but approved software, and why iPhones are more secure without an AV package (AV packages are occasionally proved to be vulnerable or unstable, and have themselves been exploited by attackers.)

But nothing's perfect. Both whitelists and signed files are still vulnerable to creative attackers who can still exploit buffer overflows in signed applications, and who use techniques like Reverse-Oriented Programming to execute malware from within existing applications.