My work around for this isn't sexy, but it does work. I'm all ears if someone else knows of a better way.
Here is what I did. So in this situation, I'm trying to filter out IIS logs (s_net is simply tcp/udp listening on port 514). So here is what is sitting in my iis.conf file, inside of conf.d
parser p_iis_pattern_db {
db_parser(file("/etc/syslog-ng/patterndb.d/strip_header.xml"));
};
filter f_iis {
match("iis", value(".classifier.class"));
};
rewrite r_iis {
set("${real_sender}", value("HOST"));
set("iis", value("PROGRAM"));
};
destination d_iis {
file("/nfs/syslog/iis/$YEAR/$MONTH/$DAY/${real_sender}/$YEAR-$MONTH-$DAY-$HOUR-${real_sender}-$PROGRAM.log"
template("${orig_sender}${msg}\n"));
};
log {
source(s_net);
parser(p_iis_pattern_db);
filter(f_iis);
rewrite(r_iis);
destination(d_iis);
flags(final);
};
Then, I have a pattern file in patterndb.d (strip_header.xml), which looks like this:
<patterndb version='3' pub_date='2017-02-07'>
<ruleset name='strip_sender' id='strip_leading_sender'>
<rules>
<rule id='iis' class='iis'>
<patterns>
<pattern>@ESTRING:datestamp: @@IPv4:real_sender@ iis - - - @ANYSTRING:msg@</pattern>
</patterns>
</rule>
</rules>
</ruleset>
</patterndb>
I would prefer to be filtering based upon a MACRO value other than the message, as I think this is probably rather expensive; it's having to introspect the message of every package. Because data is coming from a relay running an older version of syslog-ng, I perform an outbound tag of data on that relay, and the keep-hostname/chain-hostname options aren't working.
My other worry is that this is going to get sloppy in a hurry as I start adding in non-relay sources. I might have to do the whole thing where I name the conf files in a certain way to assure they are performed in the correct order, in order to save CPU cycles (is it alphabetical?).