wildfly and logrotate: wildfly still logs messages to already rotated server.log

Question

For wildfly (on linux) I need following logging scenario: daily rotation of server.log and removing old log files which are older than 90 days. I don't see a way to configure this in wildfly/log4j (the problem is here to remove old log files, but I will be happy for tips to do this directly with wildfly configuration). So I have to use linux logrotate for this. I have following logrotate configuration file:

/var/log/wildfly/capp/*.log {
    missingok
    daily
    notifempty
    rotate 90
    maxage 90
    dateext
    dateformat -%Y%m%d
}

The server.log will be rotated successfully in the early morning. But: wildfly is still writing the log messages into the already logrotated file (see at the timestamps of the last write access):

#> ls -la
-rw-r--r-- 1 wildfly-capp wildfly-capp      0  4. Apr 03:40 server.log
-rw-r--r-- 1 wildfly-capp wildfly-capp 368909  4. Apr 07:00 server.log-20190404

Is there a way to force wildfly to use the server.log file instead the already rotated file (without restarting wildfly)? Or is it possible to change the wildfly logging configuration to remove logging files which are older than x days?

The wildfly logging configuration is:

<subsystem xmlns="urn:jboss:domain:logging:5.0">
        <console-handler name="CONSOLE">
            <level name="INFO"/>
            <formatter>
                <named-formatter name="COLOR-PATTERN"/>
            </formatter>
        </console-handler>
        <file-handler name="FILE" autoflush="true">
            <formatter>
                <named-formatter name="PATTERN"/>
            </formatter>
            <file relative-to="jboss.server.log.dir" path="server.log"/>
            <append value="true"/>
        </file-handler>
        <logger category="com.arjuna">
            <level name="WARN"/>
        </logger>
        <logger category="org.jboss.as.config">
            <level name="DEBUG"/>
        </logger>
        <logger category="sun.rmi">
            <level name="WARN"/>
        </logger>
        <root-logger>
            <level name="INFO"/>
            <handlers>
                <handler name="CONSOLE"/>
                <handler name="FILE"/>
            </handlers>
        </root-logger>
        <formatter name="PATTERN">
            <pattern-formatter pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/>
        </formatter>
        <formatter name="COLOR-PATTERN">
            <pattern-formatter pattern="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/>
        </formatter>
</subsystem>

nobody · Answer 1 · 2019-04-05T05:29:35.267

After you rotate logs, you need to tell wildfly that logs are rotated and that it should start to write into new log file. Usually, that is done with HUP signal sent to the daemon, or you can just restart it. Otherwise, daemon will keep the filehandle of open file and it will write in the old file. That is done by adding postrotate section telling what needs to be done after logs are rotated. Take a look at examples for postrotate section in logrotate config file. Here are some examples from my computer for ufw and samba:

postrotate
    invoke-rc.d rsyslog rotate >/dev/null 2>&1 || true
endscript


postrotate
    if [ -d /run/systemd/system ] && command systemctl >/dev/null 2>&1 && systemctl is-active --quiet samba-ad-dc; then
        systemctl kill --kill-who all --signal=SIGHUP samba-ad-dc
    elif [ -f /var/run/samba/samba.pid ]; then
        # This only sends to main pid, See #803924
        kill -HUP `cat /var/run/samba/samba.pid`
    fi
endscript

For wildfly you will have to write your own command(s) to make wildfly reopen log files.

Thanks for your answer. But HUP signal/restarting wildfly is not what I want. But your answer opens my eyes. I understand now that logrotate is not what I need (in this case). — Steffen, Apr 05 '19 at 08:00

score 0 · Answer 2 · answered May 27 '20 at 16:02

Can you combine logrotate and JBOSS options? For standalone configuration change logging.properties:

logger.handlers=FILE
handler.log_rotation.suffix=-yyyyMMdd

And only use logrotate for files with an - in the filename:

/var/log/wildfly/capp/*.log-* {

score 0 · Answer 3 · answered Apr 05 '19 at 08:20

The point is that I don't want restarting wildfly. Thanks to "nobody" for his answer. But now I know logrotate is not right thing for my case.

A workaround is using the "size-rotating-file-handler" (instead of "file-handler") in wildfly. To switch to the size rotating handler I have to edit the standalone.xml and replacing the <file-handler name="FILE" autoflush="true"> section of <subsystem xmlns="urn:jboss:domain:logging:5.0"> with:

<size-rotating-file-handler name="FILE" autoflush="true">
            <formatter>
                <named-formatter name="PATTERN"/>
            </formatter>
            <file relative-to="jboss.server.log.dir" path="server.log"/>
            <append value="true"/>
            <rotate-size value="1M"/>
            <max-backup-index value="10"/>
</size-rotating-file-handler>

This will rotate the server.log file after the log file is greater than 1MB and will keep only 10 log backup files. All older log files will be removed.

I think if you want have a daily rotation log file appender which removes log files older than x days, you must write a own DailyRotateFileAppender which can also removes old log files. Then you must integrate your new file appender into wildfly (wildfly must find the class file and the standalone.xml configuration must be changed so that wildfly will be used the new file appender). I think this should work. For me, however, the time required to do this is too large ...

wildfly and logrotate: wildfly still logs messages to already rotated server.log

3 Answers3