How do I properly check if a program is a virus/trojan in VMware?

Question

How I should check if a program is a virus in VMware? Some programs I do need admin ability to install and it makes sense. But how do I know if it's doing more than I want? Some thoughts are:

• How many processes open when I launch the application
• What is added to the startup tab in msconfig
• If any services are added.

That's pretty much all my ideas. Even if it does something I recognize I wouldn't know if it's necessary or not. What are some rule of thumb?

-Edit- What about registries, can I use that information to help? Maybe have a scanner tell me if the application I just used has messed with sections (like bootup) it shouldn't have?

Answer 1

Run an outgoing firewall that prompts for new connections.

If the software attempts to make many outbound connections then it might be up to no good. A lot of software will check for updates either on initial start up or periodically, so you'll have to let the first one through. Don't check the "remember my answer for this application" option (one should exist) so you can see when the next time it "calls home".

This will also alert you to outgoing connection attempts from software you haven't directly launched - another sign that you've got some malware installed.

Answer 2

Wireshark to monitor traffic, Process Explorer to monitor file and registry changes. Keep a "known good" snapshot to boot into every time to reduce possible contamination. Don't give it an internet connection if you don't have to.

Answer 3

Along the lines of your analysis (though, it's always safer to check where you downloaded it from and use local antivirus software),

1. Check for what network communications it attempts.
   It's always fairly easy to enumerate what network activity is likely from the program description.
   You can use the Sysinternals TCPView to follow it or just do frequent netstat.
   Some Host antivirus/firewall tools also allow configuring a block for a process.
   • Most malware focuses on 'corrupting' other applications on your system.
     This means, just following the newly installed application will not be sufficient.
     You need a way to detect when it starts playing with other executable files in your systems too.