8

This question is for IT Pros, and people who manage a company's infrastructure. Developers should see this related answer for tools geared for them.

What are your requirements for such a Event Log Managment solution? What do you currently, or do you hope to derive from such a system?

More info:

I found a related question that focuses on how an application should save to the eventlog, but this question is orthogonal to that. Here I'd like to what tools, reports, and surrounding alerts you want from a system that aggregates, and analyzes log files.

In an Enterprise setting this would undoubtedly include: Windows Event Logs, Firewall, Router logs, syslog, etc.

Community
  • 1
makerofthings7
  • 50,090
  • 54
  • 250
  • 536
  • 7, according to your update, are you specifically looking to *exclude* application logs (the aggregation thereof, NOT *how/what to log*)? Because a SIEM that ignores that, is pretty much useless (or close to that) - lowlevel network/OS data without the logical information from the application is way to obtuse to glean any real conclusions. – AviD Nov 23 '10 at 17:44
  • Maybe I should update again... I want to include the application logs from systems across my Enterprise. I also want to centralize the monitoring and utility of the logs from a central point. My intent is to gain value from whatever information the proprietary application chooses to log. – makerofthings7 Nov 23 '10 at 17:50
  • I deleted my answer, as it is not relevant to the question anymore – Phoenician-Eagle Nov 23 '10 at 18:53
  • Big Brother http://bb4.com/ is a great example of such an enterprise network monitoring tool. – Mark S. Jun 10 '12 at 03:33
  • Can you elaborate and tell us why it is a great tool? What features or functionality did you discover?/ – makerofthings7 Jun 10 '12 at 04:11
  • You may want to add "User Experience Monitoring" to this list. Its a new area, but vital for effective system monitoring. You can have all the disk, ram, processors you want, but if it still takes 10 minues to logon, or 5 minutes top open a word processor then other info is sort of irrelavent.. –  Apr 11 '14 at 09:59

2 Answers2

10

Here are a few different features you should consider, note that you might not need all of these, and one might not be better than the other - but they are points to consider:

  • Coverage:
    • Windows Event Log on servers, all current versions
    • Windows Event Log on clients, all versions in your org (you'd be surprised how many Win98s there still are...)
    • syslog on Linux/Unix servers according to your flavors
    • syslog on Linux/Unix workstations (if you have) according to common flavors in use
    • generic SNMP traps
    • Other OSs you might have...
    • Network equipment (if thats where you want to take it): Firewalls, routers, gateways, proxies, WAFs, IDS/IPS, etc
    • Web/App server logs: e.g. IIS, Apache, WebSphere, WebLogic, etc, whatever else you have
    • Databases
    • Mail servers
    • MQ servers if you have them and they are critical
    • AD/directory/IdM servers
    • security and control servers, e.g. content filtering gateway, HPOV, MS SMS, etc.
    • etc any other infrastructure you find critical
    • Application logs! As I said in my comment up top, this are of critical importance, yet are most often neglected. While there is no real common format (there are some structures but nothing data-specific), the SIEM product/solution should:
      a. be able to receive and generically parse a variety of formats and files, e.g. CSV, XML, flatfiles, etc.
      b. ideally would be able to define specific formats and recognize the data
  • Agent vs agentless (aka push vs. pull)
    • There are advantages to each, which is better really depends on your specific environment.
    • Best would be support of both models, and use each where appropriate
    • Even bester is support of a builtin API to log directly to it.
  • Aggregation and consolidation
  • Correlation - i.e. link different events from different sources together to form a single meaningful incident. Ideally should use a variety of techniques
  • Reporting and dashboards - should be BI-like, both for trend analysis and specific incident/user investigation
  • Automated analysis and active alerting, depending on configuration. Should be flexible enough to fit your needs and expectations. Analytical engines should be able to identify certain classes of misuse/fraud/attacks in (near-)real time.
  • Historical data, and reports/analysis/correlation over time
  • If its relevant, some products offer "Compliance" packages, to assist in meeting specific regulatory requirements.
  • high availability
  • Ample protection of log data, both its confidentiality and its integrity, using combination of encryption, access control, self-logging, etc.
  • Remote management (securely, of course)
  • Possibility to plug into SOC if/when you get there
AviD
  • 72,138
  • 22
  • 136
  • 218
  • 2
    Holy feature set Batman! +1 ...this will take time to digest; however I think that you will probably add to this over time and I'll be very interested if/when you do. – makerofthings7 Nov 23 '10 at 20:14
  • :D, yup, I admit that I did not spend a ton of time ensuring this is complete, however these are things I've been thinking about / working on. A caveat, I find it doubtful that you will find a perfect solution with full coverage of all these features. As I said, these are some of the main points to consider, then do your own tradeoff of whats most important to you, according to your own environment and risk analysis. – AviD Nov 23 '10 at 20:17
  • 2
    Very nice list. As an addendum, look *very* closely at log source support. We got burned with our purchase a few years ago because of how overstated their Windows Server 2008 support really was. – Scott Pack Dec 09 '10 at 21:21
3

You can check out Anton Chuvakin's Log Management Buyer Checklist

Tate Hansen
  • 13,714
  • 3
  • 40
  • 83