Enabling Apache Zookeeper logging

 Introduction

By default zookeeper sends the log messages to CONSOLE and hence lands up in a `zookeeper.out` file in the folder from where Zookeeper is started. This file gets overwritten every time the service is restarted and hence information is lost

This article describes how logging can be enabled for Apache Zookeeper.

Note: If you are using the Zookeeper shipped with Imply, use regular Imply steps to start the zookeeper and the logs should be found under `IMPLY_SOFTWARE_DIR/var/sv/zk`.

Enabling logging for Apache Zookeeper

The settings for the Zookeeper can be set via ZOOKEEPER_PATH/conf/log4j.properties. But some of the log4j properties will be lost when `bin/zkServer.sh` is invoked, which also sets some of the variables.

Environment variables take precendence over all settings. So creating a startup script like the following  would be helpful.

zooStart.sh

export ZOOCFG=zookeeper.properties
export ZOOCFGDIR=/root/zookeeper-3.4.13/conf
export ZOO_LOG_DIR=/root/zookeeper-3.4.13/log
export ZOO_LOG4J_PROP='INFO,CONSOLE,ROLLINGFILE'
bin/zkServer.sh start

zooStop.sh

export ZOOCFG=zookeeper.properties
export ZOOCFGDIR=/root/zookeeper-3.4.13/conf
bin/zkServer.sh start

Starting the services using `zooStart.sh` would create the necessary log files in /root/zookeeper-3.4.13/log.

Log Rotation

To change the log file size and number of back log files to retain, use ZKPATH/conf/log4j.properties. Following are the important parameters:

# Max log file size of 10MB
log4j.appender.ROLLINGFILE.MaxFileSize=10MB
# uncomment the next line to limit number of backup files
log4j.appender.ROLLINGFILE.MaxBackupIndex=10

Zookeeper log file names are determined by the following parameters in the same file:

zookeeper.log.file=zookeeper.log
zookeeper.tracelog.file=zookeeper_trace.log

Logging Options

DEBUG, TRACE are the other options that can be passed via ZOO_LOG4J_PROP environment variable.  These options generate a lot more information and the log files could be big.  These modes can be used for debugging zookeeper issues.

Reference:

Zookeeper Best Practices for Imply