CentOS7 + rsyslog-elasticsearch.x86_64 + nginx = charset=utf-8] is not supported

Question

I am building a tiny system for log analysis, and not able to install additional agent, so using already working rsyslog to send logs to elasticsearch.

My rsyslog and elasticsearch are separate VM's. These VM's have 1 common network: 10.8.4.0/24 .

rsyslog server has installed rsyslog-elasticsearch.x86_64 and configured to receive error logs from all servers in all infrastructure.

My ElasticSearch is running on localhost:9200, I have setup nginx as a proxy to it.

here are my configs:

[root@all-logs rsyslog.d]# cat 00-Send2Elastic.conf
module(load="omelasticsearch")
template(name="testTemplate"
         type="list"
         option.json="on") {
           constant(value="{")
             constant(value="\"timestamp\":\"")      property(name="timereported" dateFormat="rfc3339")
             constant(value="\",\"message\":\"")     property(name="msg")
             constant(value="\",\"host\":\"")        property(name="hostname")
             constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
             constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
             constant(value="\",\"syslogtag\":\"")   property(name="syslogtag")
           constant(value="\"}")
         }
action(type="omelasticsearch"
       server="10.8.4.21"
       serverport="29200"
       template="testTemplate"
       searchIndex="test-index"
       searchType="test-type"
       bulkmode="on"
       maxbytes="100m"
       queue.type="linkedlist"
       queue.size="5000"
       queue.dequeuebatchsize="300"
       action.resumeretrycount="-1")

# http://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html
# https://sematext.com/blog/recipe-rsyslog-elasticsearch-kibana/

And Nginx:

[root@test-elk centos]# cat /etc/nginx/conf.d/elastic.conf 
server {
    listen 10.8.4.21:29200;

    server_name elastic.host;

    allow 10.8.4.0/24;
    deny all;

    location / {
        proxy_pass http://localhost:9200;
#        proxy_http_version 1.1;
#        proxy_set_header Upgrade $http_upgrade;
#        proxy_set_header Connection 'upgrade';
#        proxy_set_header Host $host;
#        proxy_cache_bypass $http_upgrade;        
    }
}

Everything else seems have no issue :slight_smile: at first look.

Error log from rsyslogd -nd: https://pastebin.com/YDrTi2q5

error line:

0478.493507244:action 0 queue:Reg/w0: omelasticsearch: pWrkrData reply: '{"error":"Content-Type header [text/json; charset=utf-8] is not supported","status":406}'

Maybe include the full error `0481.499106700:action 0 queue:Reg/w0: omelasticsearch: error in elasticsearch reply: bulkmode insert does not return array, reply is: {"error":"Content-Type header [text/json; charset=utf-8] is not supported","status":406}` — Jacob Evans, Jan 30 '18 at 04:52
As i understand bulk error means that it is a single message, so when it will be a bunddle it should pass... — BiG_NoBoDy, Jan 31 '18 at 07:23

score 0 · Accepted Answer · answered Mar 21 '18 at 22:31

0

have done an update to 8.33.1-2 from rsyslog.com repo, and everything works just fine!

answered Mar 21 '18 at 22:31

BiG_NoBoDy

138
1
8

CentOS7 + rsyslog-elasticsearch.x86_64 + nginx = charset=utf-8] is not supported

1 Answers1