Our logs are sent from our applications to rsyslog running on the same host. Rsyslog then forwards the messages to Sumo Logic.
We need to add some metadata to our log messages in the structured data field. Some of our applications already use structured data, so we can't simply replace the structured data property in our template.
Also, the %STRUCTURED-DATA% property includes the opening and closing brackets, so we can't just put something like [%STRUCTURED-DATA% newmetadata] in the template.
According to the property replacer documentation, our options are to use FromChar and ToChar or regular expressions. I checked the source and confirmed that ToChar can't count backwards from the end.
I used the rsyslog regex tool to create the following template:
template(name="metadata_syslog" type="string" string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% [%STRUCTURED-DATA:R,ERE,1,FIELD:\[([^]]*)\]--end% extrafield=value] %msg%\n")
From the following example event
<142>1 2016-03-31T17:30:20.007Z some.host.name service/prod/app/foo_v2 - Audit [mdc@xxxxx category="io.service.segment.IndexIO$DefaultIndexIOHandler" thread="foo_v2-incremental-persist"] Processing file[dim_device.drd]
the regex tool correctly parsed out the structured data without the brackets.
When I used this template in rsyslog, I get a syntax error about the %PRI% part(debug output):
Reading a token: 9936.286569660:main thread : Called LogMsg, msg: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '"' in object definition - is there an invalid escape sequence somewhere?
rsyslogd: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '"' in object definition - is there an invalid escape sequence somewhere? [v8.17.0 try http://www.rsyslog.com/e/2207 ]
9936.286590559:main thread : Called LogMsg, msg: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '<' in object definition - is there an invalid escape sequence somewhere?
rsyslogd: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '<' in object definition - is there an invalid escape sequence somewhere? [v8.17.0 try http://www.rsyslog.com/e/2207 ]
9936.286606008:main thread : Called LogMsg, msg: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '%' in object definition - is there an invalid escape sequence somewhere?
rsyslogd: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: invalid character '%' in object definition - is there an invalid escape sequence somewhere? [v8.17.0 try http://www.rsyslog.com/e/2207 ]
Next token is token NAME ()
9936.286632522:main thread : Called LogMsg, msg: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: syntax error on token 'PRI'
rsyslogd: error during parsing file /etc/rsyslog.d/21-logging.conf, on or before line 4: syntax error on token 'PRI' [v8.17.0 try http://www.rsyslog.com/e/2207 ]
Error: popping token '=' ()
Stack now 0 1 5 28 52
Error: popping token NAME ()
Stack now 0 1 5 28
Error: popping nterm nvlst ()
Stack now 0 1 5
Error: popping token BEGIN_TPL ()
Stack now 0 1
Error: popping nterm conf ()
Stack now 0
Cleanup: discarding lookahead token NAME ()
Stack now 0
9936.286780810:main thread : Called LogMsg, msg: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'.
rsyslogd: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [v8.17.0 try http://www.rsyslog.com/e/2207 ]
