3

I installed a preview version of Kafka 3.0 with log4j2 support (http://home.apache.org/~dongjin/post/apache-kafka-log4j2-support/) on a RHEL 8 server. Kafka and Zookeeper are running successfully as systemd user services. I set the environment variable KAFKA_LOG4J_OPTS="-Dlog4j.configurationFile=file:/home/username/kafka/bin/../config/log4j2.properties" in the systemd unit file so that log4j 2.17 ist used.

However, there is one strange thing: all log4j logs are stored in a directory literally called ${kafka.logs.dir} in the home directory. The garbage collector successfully stores logs in the correct log location ~/kafka/logs. So the directory ~/kafka/logs contains files like kafkaServer-gc.log.0.current, while the hilariously strange directory ~/${kafka.logs.dir} contains server.log, controller.log, etc.

When looking at the processes for kafka and zookeeper, I can see that they both have the argument -Dkafka.logs.dir=/home/username/kafka/bin/../logs. Shouldn't this define the environment variable kafka.logs.dir, which is used in the file config/log4j2.properties? How come that Zookeeper and Kafka apparently don't have access to this $kafka.logs.dir environment variable?

Cabbage Parachute
  • 33
  • 5
  • Result of disabled pattern substitution due to Log4shell fixes? – AlexD Dec 23 '21 at 14:24
  • You're probably right. I added a definition of kafka.logs.dir to the beginning of log4j2.properties with the directory path and it still used `${kafka.logs.dir}` as directory name. – Cabbage Parachute Dec 29 '21 at 07:44

2 Answers2

2

Just add sys: prefix to the kafka.logs.dir system variable in the log4j2.properties file. That is:

  • AS-IS: ${kafka.logs.dir}
  • TO-BE: ${sys:kafka.logs.dir}

DISCLAIMER: As you can see in my name, I wrote this code (log4j2 migration of Apache Kafka) and am maintaining the preview build. The fix will be released soon. Sorry for being late.

Lee Dongjin
  • 36
  • 2
0

I have followed the steps and am able to upgrade the log4j to logj2, however, I'm having issues with controller.log which is not getting appended. Could you please help?

San
  • 1