16

You have central logging going, detailed app logging/alerting (e.g. modsec), network based security alerting (e.g. snort), and whatever else feeding your observation deck.

Do you have any cool techniques you’d like to share for how you relate security events?

How about tools? (in-house is fine, just describe what it does)

Scott Pack
  • 15,167
  • 5
  • 61
  • 91
Tate Hansen
  • 13,714
  • 3
  • 40
  • 83
  • Is this any different from [this question](http://security.stackexchange.com/q/1008/33)? Sounds duplicate... – AviD Dec 15 '10 at 11:08
  • @AviD: I disagree. I think the other question is about finding interesting events. This question is about correlating information from multiple systems. It'd be cool to get @Tate's input, but I think this question should be left open. –  Dec 15 '10 at 12:52
  • @Graham, I see your point. Though I would argue that "finding interesting events" *must* include some correlation, too... But I see why the questions should be seperate. – AviD Dec 15 '10 at 13:02
  • @Graham, you're exactly right. For example, if my log IDS (e.g. OSSEC) independently fires an “interesting” alert then I may check for correlating events/alerts from my other systems (e.g. modsec/snort/netflow/file integrity/etc.) or automate the retrieval of additional data. This gets into SEM territory, of course, but there are 1000s of ways to do this and many in-house teams have created some very cool tools/scripts to accelerate incident investigations. Answers could be simple scripts, an algorithm, fancy machine learning techniques, or anything. – Tate Hansen Dec 16 '10 at 01:16
  • 1
    cool, I'm convinced. Looking forward to seeing the answers... Just a comment, very often when these get implemented (either inhouse or with product) they focus *too* much on the infrastructure - OS, network, etc. - at the expense of (or neglecting altogether) the actual *business applications* that house the real information. – AviD Dec 16 '10 at 08:52
  • It's Q2-2015 and most of these answers are still relevant, seeing shops running RSA Security Analytics, Splunk, and OSSIM, as well as QRadar and certainly ArcSight and Trustwave. What I don't see are modernized additions such as Cybersponse, Squrrl, etc. Please speak up! – atdre May 13 '15 at 21:33

3 Answers3

5

Enterprise tools which correlate related security and logs events are generally called Security Information and Event Management (SIEM) systems. Most are designed to accept data from common log formats, IDS alerts, antivirus, firewall rule changes etc.

Other tools that I know about:

  • OSSIM, The Open Source SIEM
    http://sourceforge.net/projects/os-sim/
    One of my interns has played around with this, with generally favorable reviews. For a project, or a small shop, this is probably your best bet.

  • Intellitactics (aparrently now owned by trustwave)
    http://www.intellitactics.com/int/
    They were another big player when SEIM first started, they were unique in that they were using a text database (as opposed to Oracle like ArcSight and the rest). I heard they were going to stop using a text db and move to something else, but never heard anything from then in a long while.

  • RSA enVision
    http://www.rsa.com/node.aspx?id=3170
    This was regarded as a pile of crap, talking to the new product manager of the team at defcon, they were re-banging the product, using business intelligence tools and techniques (column store databases, etc). What I was told was EMC (the parent company of RSA) bought Greenplum (a ZFS based BI system) for this project. I would be watching them closely, and maybe get under an NDA and get the real story.

I'm using older information here, would love to hear whats correct and whats not.

chuacw
  • 175
  • 10
Reiger
  • 91
  • 2
3

The above link lists a few tools which are really going to stick out above others, such as that

  1. Novell Sentinel Log Manager 25 appears to be better than Splunk if you are starting out and aren't funded by a Global 200 company
  2. Q1Labs appears to be attempting to compete with AlienVault (OSSIM) by providing a mid-market solution that even startups and companies considering SIEM on-trial will want to leap to
  3. log2timeline implements a good concept not usually found in IDR methodologies or any commercial/free tools that I've seen in that it can specifically pinpoint gaps in time that could have been due to log file deletion (purposeful or not)

I am partial to Suricata over snort and I do not currently like any of the existing vulnerability management systems (e.g. OpenVAS, Arachni) or application monitoring systems (e.g. ModSecurity, AppSensor) -- however I am very likely to implement these along with OSSEC via OSSIM given the modern limitations of IT/Ops and AppDev shops.

It also doesn't appear that old school tools such as Beltane or Cerebus have kept up to today's needs. I guess the new freshness would be adopting integration for OSSIM and OSSEC in a Vagrant environment.

atdre
  • 18,885
  • 6
  • 58
  • 107
  • This paper mentions using SEC along with RT and a few other tools -- http://www.sans.org/reading_room/whitepapers/incident/creating-siem-incident-response-toolkit-open-source-tools_33689 – atdre Jul 06 '11 at 16:09
0

Here, we're switching from Intellitactics to Splunk, which seems to be the up-and-coming contender in the area. One of the nice things about Splunk is its extensibility; it's pretty easy to write data-mining or reporting modules for it. Another is the built-in "regex wizard" that lets you build a regex which corresponds to selected example text.

I personally have grand plans which seem to be perpetually in the "almost started" phase to use machine learning techniques for multiclass classification and clustering to organize the interesting-but-non-critical stuff that comprises 99% of logged data.

user502
  • 3,261
  • 1
  • 22
  • 18