8

I'm building a log analyser service to start monitoring mainly our pfSense Firewalls, XenServer Hypervisors, FreeBSD/Linux servers and Windows servers.

There's a lot of documentation on the internet about the ELK stack and how to make it work nicely. But I would like to use it in a different manner, but I don't know if it's a good solution or just a waste of time/disk space.

I already have a FreeBSD 10.2 machine acting as a remote syslog server, and my ideia is to simply concentrate all the logs on this machine and them the syslog server forwards the logs with logstash-forwarder to the ELK server.

It's clear to me that this approach will raise the disk requirements for this setup, but in other hand I will have only one machine with the logstash-forwarder daemon installed, which seems good to me.

But talking about problems. The logstash parser matches [host] with the hostname of the server sending the log messages, and in this approach there's only on "server" show on ELK, the remote syslog server.

I'm aware that I can customize the settings on the logstash configuration files but I don't know (and I don't have the experience to know) if this is just a simple setting on the parsers of it if will compromise the entire ELK experience.

In the end I just want some advices about my logging architecture and if it will work, or if I should go without other option.

Thanks in advance,

Vinícius Ferrão
  • 5,400
  • 10
  • 52
  • 91
  • I'm sure what you want to do is possible, but details about what format you're logging in on the central syslog server, the logging rate, etc would be great. Also, asking a specific question will get you much better responses than simply saying "I'm lost, please help". – GregL Nov 24 '15 at 01:42
  • Have you considered graylog? You can use the graylog syslog input to ingest syslog into elasticsearch. From there you can use Kibana and graylog has a syslog/pfsense extractor available. – gm3dmo Nov 24 '15 at 06:28
  • Logstash also has a [syslog](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html) input... – GregL Nov 24 '15 at 12:51

1 Answers1

3

Yes. It is possible to change the host field in logstash output with ruby filter without much hassle.

    ruby {
            code => "
                    event['host'] = event['message'].split(' ')[3]
                   "
    }

Here I assumed in the syslog server logs, the host field is the fourth field where white space is the separator.

jijinp
  • 368
  • 2
  • 5