Kafka logfile location

Question

Just to make some things clear: I'm talking about the process logfile that contains the stdout and stderr messages.

This is my systemd unit file:

[Unit]
Description=Apache Kafka server
Documentation=http://kafka.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
PIDFile=/var/run/kafka.pid
User=kafka
Group=kafka
Environment="JAVA_HOME=/usr/lib/jvm/jre"
Environment="KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:/var/log/kafka"

ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-failure
SyslogIdentifier=kafka

[Install]
WantedBy=multi-user.target

Note that I added the KAFKA_LOG4J_OPTS environment variable.

However, that doesn't seem to do anything. This is the output, when I try to start my service:

Feb 11 00:55:30 kafka01 kafka[4047]: mkdir: cannot create directory ‘/opt/kafka/bin/../logs’: Permission denied
Feb 11 00:55:30 kafka01 kafka[4047]: OpenJDK 64-Bit Server VM warning: Cannot open file /opt/kafka/bin/../logs/kafkaServer-gc.log due to No such file or directory
Feb 11 00:55:30 kafka01 kafka[4047]: log4j:WARN No appenders could be found for logger (kafka.Kafka$).
Feb 11 00:55:30 kafka01 kafka[4047]: log4j:WARN Please initialize the log4j system properly.

So, can please someone tell me the correct way to change the location of the kafka log file? KAFKA_LOG4J_OPTS doesn't work, KAFKA_LOG_DIR doesn't work either (=I was hoping kafka implemented this similar to zookeeper). The kafka documentation doesn't tell me either.

Answer 1

For other unfortunate lads like my, you need to modify LOG_DIR environment variable (tested for Kafka v0.11).

If you open script kafka-server-start or /usr/bin/zookeeper-server-start, you will see at the bottom that it calls kafka-run-class script. And you will see there that it uses LOG_DIR as the folder for the logs of the service (not to be confused with kafka topics data).

Answer 2

As eddyP23 mentioned in previous response... kata-run-class uses LOG_DIR

However there is logic in it to use Script directly parent folder if LOG_DIR variable is not set.

So if your scripts are in kafka/bin folders where you are starting kafka... then logs will be present in kafka/log

Answer 3

Maybe not actually the answer to the questions but I've arrived here searching for where kafka stores application logs, not producers/consumers data. IMO calling this data logs is really bad naming.

I've deployed a Kafka cluster on GCP Kubernetes engine and I wanted to view the logs of the application and here is how I did it.

1- Get the pods

    > kubectl get pods
    NAME      READY   STATUS    RESTARTS   AGE
    kafka-0   1/1     Running   0          32m
    kafka-1   1/1     Running   0          32m
    kafka-2   1/1     Running   0          32m
    zk-0      1/1     Running   0          23h
    zk-1      1/1     Running   0          23h
    zk-2      1/1     Running   0          23h

2- Connect to one of the Kafkas pods with interactive command

> kubectl exec -it kafka-0 /bin/bash 

3- Echo $, in command line type echo $ and press the tab button several times ... you will get a list of all defined environment variables, search for $KAFKA_HOME or just type

> cd $KAFKA_HOME 

You will get to:

/opt/kafka$ ls -al 
total 68
drwxr-xr-x 1 kafka kafka  4096 Aug  2 09:37 .
drwxr-xr-x 1 root  root   4096 Jun 26  2017 ..
-rw-r--r-- 1 kafka kafka 28824 Apr 21  2017 LICENSE
-rw-r--r-- 1 kafka kafka   336 Apr 21  2017 NOTICE
drwxr-xr-x 1 kafka kafka  4096 Jun 26  2017 bin
drwxr-xr-x 1 kafka kafka  4096 Jun 26  2017 config
drwxr-xr-x 1 kafka kafka  4096 Jun 26  2017 libs
drwxr-xr-x 2 kafka kafka  4096 Aug  2 09:37 logs
drwxr-xr-x 1 kafka kafka  4096 Jun 26  2017 site-docs

The log directory is what I was looking for ...