I'm setting up a ELK cluster using Centos 8 and version 7.4 of Elasticsearch, Logstash and Kibana. My issue is with Logstash not picking up the events coming through syslog. Configuring Logstash to read from files and sending it to elasticsearch it works fine.
An overview of the infra, the 3 systems are installed in this server.
I have SELinux disabled to make sure it's not affecting it. Logstash config is using port 5144 and I can see that it's working fine with:
[root@elk-1 conf.d]# netstat -tulpn | grep 5144
udp 0 0 0.0.0.0:5144 0.0.0.0:* 15838/java
For firewall rules I'm using the following (firewalld):
firewall-cmd --set-default-zone=internal
firewall-cmd --permanent --zone=internal --add-port=514/tcp #syslog port
firewall-cmd --permanent --zone=internal --add-port=514/udp #syslog port
firewall-cmd --permanent --zone=internal --add-port=5514/tcp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5514/udp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5600/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=5601/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=9600/tcp #logstash
firewall-cmd --permanent --zone=internal --add-port=9200/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=9300/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=80/tcp #http
firewall-cmd --permanent --zone=internal --add-port=443/tcp #https
firewall-cmd --zone=internal --add-forward-port=port=514:proto=udp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-forward-port=port=514:proto=tcp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-masquerade --permanent
So I'm redirecting the traffic from 514 to 5144. I can see the traffic comming through port 514 with tcpdump. However I don't see any traffic on port 5144. Checking the Logstash logs doesn't have anything helpful even on debug mode.
The input section of my logstash is:
input {
udp {
port => 5144
type => syslog
}
}
and output:
output {
elasticsearch {
hosts => [ "10.248.1.31:9200" ]
manage_template => false
index => "fgt-%{+YYYY.MM.dd}"
}
}
* UPDATE *
Setting Logstash to use root instead of logstash user and the config file to listen to 514 it worked. However I know that's not the best idea and would like a better fix for that.
Any ideas on why Logstash is not picking up the traffic from syslog ?
Thanks