How dangerous is it to use third-party security audit tools?

Question

This question is inspired by two related questions (How safe is backtrack to use, and How to Install, Configure and Use LSAT on Unix SE).

Besides a dated (and patched) vulnerability in LSAT (CVE-2007-1500) I have searched through NVD/CVE and found a vulnerability in Backtrack: CVE-2012-0054.

My question is whether I am right in requiring a security specialist doing an audit of my systems not to run any automated third-party tools on any of my systems (even if their source code is publicly available and they are compiled from source on site).

Answer 1

Security testing implies risks.

Most security tools have warnings and disclaimers because they pose risks to systems they target. A security specialist would understand how using a particular tool or intrusive test can negatively impact the target systems but undesirable outcomes and accidents can happen. For example, a classic ‘OR ‘1’=’1 SQL injection in a DELETE query could very easily delete all the contents of the table.

Another example is NMAP scanner containing auditing script that are marked safe or intrusive. Some scripts are intrusive because they use significant resources on the remote system, are likely to crash the system or service, or are likely to be perceived as an attack by the remote administrators.

Auditing is a rather passive activity focused on observing rather than interacting with the target. Backtrack and other toolboxes focus more on active interaction that is typical of penetration tests.

Answer 2

The thing is that automated tools sometimes are nescesary to perform an audit. If you do not want to run these automated tools (often scripts that pull some basic info about software and the underlying OS) then you will have to deliver everything manually and your auditor will be standing right next you. Since this will increase the time spent for the auditor you can expect the price to go up compared to a standard audit. So if you are prepared to pay about 2 to 3 times the normal price, then there will probably be no issues.

However when we are talking about a security audit from a blackbox point of view (this is more of a pentest than an audit really) you can expect him to use some basic automated tools, for instance nmap, to try and get an understanding of the network. If the person has to write his own script, you can consider that to be an automated tool as well in principle. So that would mean the pentester will have to check everything manually, at around 8 seconds to execute and evaluate a ping, you can guess how long it would take just to see what hosts are online in a basic /24 subnet.

There are other reasons as well why auditors use automated tools, the tool doesn't forget to check anything, an auditor on the other hand might overlook some detail and forget to check a certain parameter.

So yea you can forbid them from executing automated tools (I'm talking a basic script running on your system) or using Backtrack with nmap on your network, but chances are you will end up paying a lot more and covering a lot less. Most of tools like BackTrack have been tested and used by many professionals (see the question about Backtrack). So these tools are pretty safe to run.

What you should make sure to put in your contract, is that the security audit cannot interfere or cause down time on your production systems. (there are some exceptions, but they are in a situation where there is no other way)

On your exploit

That exploit requires someone to be locally logged on the system, so he will need to get access to a validly defined account on the system. Since most people use backtrack from a life cd rather than using an install, chances of this having an impact on your network is really low. Further more this is exploit is likely to be present on systems similar to backtrack, like Debian or Ubuntu. These types of exploits are present in most software, probably even the software you are using to run your business.