Are there any log analyzers that are using Bayesian algorithms or other learning algorithms? I found btail but the version number (0.2) doesn't give a good outlook.
Asked
Active
Viewed 813 times
4
-
btail is the only one I'm aware of - Note that it's not an *analyzer* - it's a *filter*. What are you trying to accomplish? Bayesian filtering is a powerful tool, but with more information someone may be able to point you at a more appropriate tool that requires less initial work to get going. – voretaq7 Oct 08 '10 at 19:05
-
I'm looking for filter/analyzer which I could teach what log entry is important and what is not and then send appropriate alerts if it's critical. – raspi Oct 09 '10 at 12:12
2 Answers
4
You might check out crm114
. It's commonly used for spam, but can be directed at other stuff as well, like information firewalling. It's installable in Debian:
Description: versatile classifier for e-mail and other data
CRM114, the Controllable Regex Mutilator, is a system to examine incoming
e-mail, system log streams, data files, or other data streams, and to sort,
filter, or alter the incoming files or data streams however the user
desires. Criteria for categorization of data can be by satisfaction of
regular expressions, by sparse binary polynomial matching with a Bayesian
Chain Rule evaluator, or by other means.
.
CRM114 is not just another drop-in spam-filtering system; its Sparse
Binary Polynomial Hashing methods give it the power to develop highly
accurate Bayesian filters on very little training.
.
CRM114 is compatible with SpamAssassin or other spam-flagging software; it
can also be pipelined in front of or behind procmail. CRM114 is also useful
as a syslog or firewall log filter, to flag up important events but ignore
the ones that aren't meaningful.
.
For mail filtering, installing metamail or mew-bin packages is
recommended in order to have tools to decode MIME attachments.
Homepage: http://crm114.sourceforge.net
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
![](../../users/profiles/919.webp)
jldugger
- 14,122
- 19
- 73
- 129
-
Thanks for mentioning this. I ended up spending nearly all of my day working with it. – Brian Cline Oct 29 '13 at 19:35
1
I am searching for a similar tool that will help me sort the logs and I've found SEC and Splunk. I did not try any of them, but it might be helpful for what you need.
![](../../users/profiles/30946.webp)
Mircea Vutcovici
- 16,706
- 4
- 52
- 80