Bayesian / learning log analyzer?

Question

Are there any log analyzers that are using Bayesian algorithms or other learning algorithms? I found btail but the version number (0.2) doesn't give a good outlook.

btail is the only one I'm aware of - Note that it's not an *analyzer* - it's a *filter*. What are you trying to accomplish? Bayesian filtering is a powerful tool, but with more information someone may be able to point you at a more appropriate tool that requires less initial work to get going. — voretaq7, Oct 08 '10 at 19:05
I'm looking for filter/analyzer which I could teach what log entry is important and what is not and then send appropriate alerts if it's critical. — raspi, Oct 09 '10 at 12:12

score 4 · Accepted Answer · answered Dec 22 '10 at 05:39

You might check out crm114. It's commonly used for spam, but can be directed at other stuff as well, like information firewalling. It's installable in Debian:

Description: versatile classifier for e-mail and other data
 CRM114, the Controllable Regex Mutilator, is a system to examine incoming
 e-mail, system log streams, data files, or other data streams, and to sort,
 filter, or alter the incoming files or data streams however the user
 desires. Criteria for categorization of data can be by satisfaction of
 regular expressions, by sparse binary polynomial matching with a Bayesian
 Chain Rule evaluator, or by other means.
 .
 CRM114 is not just another drop-in spam-filtering system; its Sparse
 Binary Polynomial Hashing methods give it the power to develop highly
 accurate Bayesian filters on very little training.
 .
 CRM114 is compatible with SpamAssassin or other spam-flagging software; it
 can also be pipelined in front of or behind procmail. CRM114 is also useful
 as a syslog or firewall log filter, to flag up important events but ignore
 the ones that aren't meaningful.
 .
 For mail filtering, installing metamail or mew-bin packages is
 recommended in order to have tools to decode MIME attachments.
Homepage: http://crm114.sourceforge.net
Bugs: https://bugs.launchpad.net/ubuntu/+filebug

Thanks for mentioning this. I ended up spending nearly all of my day working with it. — Brian Cline, Oct 29 '13 at 19:35

score 1 · Answer 2 · answered Dec 22 '10 at 14:39

1

I am searching for a similar tool that will help me sort the logs and I've found SEC and Splunk. I did not try any of them, but it might be helpful for what you need.

answered Dec 22 '10 at 14:39

Mircea Vutcovici

16,706
4
52
80

Bayesian / learning log analyzer?

2 Answers2