0

I'm setting up a ELK cluster using Centos 8 and version 7.4 of Elasticsearch, Logstash and Kibana. My issue is with Logstash not picking up the events coming through syslog. Configuring Logstash to read from files and sending it to elasticsearch it works fine.

An overview of the infra, the 3 systems are installed in this server.

I have SELinux disabled to make sure it's not affecting it. Logstash config is using port 5144 and I can see that it's working fine with:

[root@elk-1 conf.d]# netstat -tulpn | grep 5144
udp        0      0 0.0.0.0:5144            0.0.0.0:*                           15838/java

For firewall rules I'm using the following (firewalld):

firewall-cmd --set-default-zone=internal
firewall-cmd --permanent --zone=internal --add-port=514/tcp #syslog port
firewall-cmd --permanent --zone=internal --add-port=514/udp #syslog port
firewall-cmd --permanent --zone=internal --add-port=5514/tcp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5514/udp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5600/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=5601/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=9600/tcp #logstash
firewall-cmd --permanent --zone=internal --add-port=9200/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=9300/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=80/tcp #http
firewall-cmd --permanent --zone=internal --add-port=443/tcp #https
firewall-cmd --zone=internal --add-forward-port=port=514:proto=udp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-forward-port=port=514:proto=tcp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-masquerade --permanent

So I'm redirecting the traffic from 514 to 5144. I can see the traffic comming through port 514 with tcpdump. However I don't see any traffic on port 5144. Checking the Logstash logs doesn't have anything helpful even on debug mode.

The input section of my logstash is:

input {
  udp {
    port => 5144
    type => syslog
  }
}

and output:

output {
  elasticsearch {
    hosts => [ "10.248.1.31:9200" ]
    manage_template => false
    index => "fgt-%{+YYYY.MM.dd}"
    }
}

* UPDATE *

Setting Logstash to use root instead of logstash user and the config file to listen to 514 it worked. However I know that's not the best idea and would like a better fix for that.

Any ideas on why Logstash is not picking up the traffic from syslog ?

Thanks

Adonist
  • 267
  • 2
  • 9
  • 1
    Have you tried echoing log lines (or any text really) directly to the ports in question with `nc` or `telnet`? Something like `nc localhost 5144` for TCP, or `nc --udp localhost 5144` for UDP. – GregL Nov 20 '19 at 13:03
  • Hi Greg, yeah in fact I did: `echo 'Hellow World' | nc --udp elk-1.ammeon.com 5144` and with 514. I can see that in tcp dump, but not on Kibana. So logstash is not sending it somehow. Also, if I do 514 and do a tcpdump on 5144 I don't see anything. I would expect to see since it's being redirect right ? – Adonist Nov 20 '19 at 15:32
  • So nothing is showing up in Kibana even when you NC it directly to 5144? Have you run LS with the stdout output, to see if your pipeline is working? I've also found that the `syslog` type for inputs doesn't work as well as I'd like. I'm using TCP, but I don't have a `type` set. – GregL Nov 20 '19 at 15:47
  • I've updated the message. It didn't work. setting for stdout wasn't working either. I configured logstash to run as root and now it worked, however that's not the best solution anyway. – Adonist Nov 20 '19 at 16:03
  • 1
    Perhaps try a plain UDP input with a stdout output, to see if you can narrow down the cause of the issues. Also, run Logstash interactively while running through the config iterations so that you can see what it's doing each time. – GregL Nov 20 '19 at 16:06

0 Answers0