I am building a tiny system for log analysis, and not able to install additional agent, so using already working rsyslog to send logs to elasticsearch.
My rsyslog and elasticsearch are separate VM's. These VM's have 1 common network: 10.8.4.0/24 .
rsyslog server has installed rsyslog-elasticsearch.x86_64 and configured to receive error logs from all servers in all infrastructure.
My ElasticSearch is running on localhost:9200, I have setup nginx as a proxy to it.
here are my configs:
[root@all-logs rsyslog.d]# cat 00-Send2Elastic.conf
module(load="omelasticsearch")
template(name="testTemplate"
type="list"
option.json="on") {
constant(value="{")
constant(value="\"timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"syslogtag\":\"") property(name="syslogtag")
constant(value="\"}")
}
action(type="omelasticsearch"
server="10.8.4.21"
serverport="29200"
template="testTemplate"
searchIndex="test-index"
searchType="test-type"
bulkmode="on"
maxbytes="100m"
queue.type="linkedlist"
queue.size="5000"
queue.dequeuebatchsize="300"
action.resumeretrycount="-1")
# http://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html
# https://sematext.com/blog/recipe-rsyslog-elasticsearch-kibana/
And Nginx:
[root@test-elk centos]# cat /etc/nginx/conf.d/elastic.conf
server {
listen 10.8.4.21:29200;
server_name elastic.host;
allow 10.8.4.0/24;
deny all;
location / {
proxy_pass http://localhost:9200;
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection 'upgrade';
# proxy_set_header Host $host;
# proxy_cache_bypass $http_upgrade;
}
}
Everything else seems have no issue :slight_smile: at first look.
Error log from rsyslogd -nd: https://pastebin.com/YDrTi2q5
error line:
0478.493507244:action 0 queue:Reg/w0: omelasticsearch: pWrkrData reply: '{"error":"Content-Type header [text/json; charset=utf-8] is not supported","status":406}'