Forwarding rsyslog to syslog-ng, with FQDN and facility separation

Question

I'm attempting to configure my rsyslog clients to forward messages to my syslog-ng log repository systems. Forwarding messages works "out of the box", but my clients are logging short names, not FQDNs. As a result the messages on the syslog repo use short names as well, which is a problem because one can't determine which system the message originated from easily. My clients get their names through DHCP / DNS.

I've tried a number of solutions trying to get this working, but without success. I'm using rsyslog 4.6.2 and syslog-ng 3.2.5.

I've tried setting $PreserveFQDN on as the first directive in /etc/rsyslog.conf (and restarting rsyslog of course). It seems to have no effect.

hostname --fqdn on the client returns the proper FQDN, so the problem isn't whether the system can actually figure out its own FQDN.

$LocalHostName <fqdn> looked promising, but this directive isn't available in my version of rsyslog (Available since 4.7.4+, 5.7.3+, 6.1.3+). Upgrading isn't an option at the moment.

Configuring the syslog-ng server to populate names based on reverse lookups via DNS isn't an option. There are complexities with reverse DNS and the public cloud.

Specifying for the forwarder to use a custom template seems like a viable option at first glance. I can specify the following, which causes local logging to begin using the FQDN on the syslog-ng repo.

$template MyTemplate, "%timestamp% <FQDN> %syslogtag%%msg%"
$ActionForwardDefaultTemplate MyTemplate

However, when I put this in place syslog-ng seems to be unable to categorize messages by facility or priority. Messages come in as FQDN, but everything is put in to user.log. When I don't use the custom template, messages are properly categorized under facility and priority, but with the short name.

So, in summary, if I manually trick rsyslog into including the FQDN, priority and facility becomes lost details to syslog-ng. How can I get rsyslog to do FQDN logging which works properly going to a syslog-ng repository?

rsyslog client config:

$ModLoad imuxsock.so    # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously done by rklogd)
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 *
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
$WorkDirectory /var/spool/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
*.* @syslog-ng1.example.com
*.* @syslog-ng2.example.com

syslog-ng configuration (abridged for brevity):

options {
    flush_lines (0);
    time_reopen (10);
    log_fifo_size (1000);
    long_hostnames (off);
    use_dns (no);
    use_fqdn (yes);
    create_dirs (no);
    keep_hostname (yes);
};
source src {
    unix-stream("/dev/log");
    internal();
    udp(ip(0.0.0.0) port(514));
};
destination per_host_destination {
file( "/var/log/syslog-ng/devices/$HOST/$FACILITY.log" owner("root") group("root")  perm(0644) dir_owner(root) dir_group(root) dir_perm(0775) create_dirs(yes));
};
log { source(src); destination(per_facility_destination); };

Upgrading isn't an option at the moment. <- No luck because $LocalHostName is really the better solution to this problem IMHO. Rsyslog has this feature both in RHEL5 and 6. — , Oct 30 '13 at 14:41

score 2 · Accepted Answer · answered Oct 31 '13 at 16:25

Okay, I found a way. It turns out the $ActionForwardDefaultTemplate by default is set to :

$ActionForwardDefaultTemplate RSYSLOG_ForwardFormat

Per this rsyslog documentation, the RSYSLOG_ForwardFormat is specifically used to maintain interoperability between different syslogs. No shocker then that if you modify this Forward template as I described originally, some functionality for other syslogs break:

$template MyTemplate, "%timestamp% <FQDN> %syslogtag%%msg%"
$ActionForwardDefaultTemplate MyTemplate

The workaround I found was to dig up the back-end template that RSYSLOG_ForwardFormat uses and mimic it. After digging through the source I eventually found that RSYSLOG_ForwardFormat is actually this:

"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"

So, if I create a custom template with almost the same content, but substitute my FQDN in place of the %HOSTNAME% macro, syslog-ng facility separation works properly and the system logs with FQDN:

$template MyForwardTemplate, "<%PRI%>%TIMESTAMP% <fqdn> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
$ActionForwardDefaultTemplate MyForwardTemplate

Forwarding rsyslog to syslog-ng, with FQDN and facility separation

1 Answers1