How do you de-duplicate security risk findings obtained from various different tools?

Question

You potentially use a variety of scanners and processes (e.g. Threat Modelling) which produces a set of overlapping outputs. How do you avoid repeated findings which are duplicated across toolsets?

Answer 1

I have written software to do this in the past.

Each tool needs a parser (you could use an API client as well) that processes tool output into a table with these columns:

• Target ID
• Tool ID (e.g. "nessus")
• Tool vulnerability ID - whatever ID the tool gives to the test
• Info - Freeform text of tool output

My software had a UI and you defined your own knowledge base of vulnerabilities.

• Vuln ID
• Title
• Severity
• Description

To connect these together, you also defined mappings:

• (Tool ID, Tool vulnerability ID) -> Vuln ID

So, the first time you imported results from a particular tool, all the results would come up as "unmapped". You'd then manually define mappings for all of them. If you imported future results from that tool, you'd benefit from mappings already defined - but typically you'd have to map a few new IDs that hadn't triggered before.

In some cases, manually mapping became tedious so I developed a heuristic automap that worked off IDs like MS bulletin numbers and CVEs. But this always needed a little bit of manual oversight.

Once you've got the findings mapped, you can look at a list of all the vulnerabilities, and see which tool reported on them. I usually took that approach that if two tools agreed on a finding, I would take that as confirmation. If only did, I've investigate manually - and the raw tool output was only a click away. Over time using a system like this you learn the intricacies of different tools.

I know that Canopy is commercial software that uses a similar approach, and I expect there are others.