Clean Up Your Zookeeper History

TL;DR: Use zkCleanup.sh -n 5 command:

By default, Zookeeper does not delete any of its history to allow recovery. This however can lead to disk space issues over long-ish periods of time. Therefore, it is advisable to clean up once in a while.

Update: I found out that /usr/share/zookeeper/bin contains a script named zkCleanup.sh. You can use it to remove the old logs:

# In your zookeeper install directory
# (usually /usr/share/zookeeper):
#
./zkCleanup.sh -n 5

It basiacally executes the command line below:

sudo \
java -cp zookeeper.jar:log4j.jar:slf4j-api.jar:conf \
    org.apache.zookeeper.server.PurgeTxnLog \
    /var/lib/zookeeper \
    -n 5

The manual way

My configuration is:

• The database is located on /var/lib/zookeeper

  15M -rw-r--r-- 1 zookeeper zookeeper 65M Oct 20 13:09 log.96633d
  16K -rw-r--r-- 1 zookeeper zookeeper 65M Oct 20 13:11 log.96b52f
  8.0K -rw-r--r-- 1 zookeeper zookeeper 65M Oct 20 13:11 log.96b55b
  62M -rw-r--r-- 1 zookeeper zookeeper 65M Oct 24 16:17 log.96b568
  64M -rw-r--r-- 1 zookeeper zookeeper 65M Oct 28 21:44 log.981574
  38M -rw-r--r-- 1 zookeeper zookeeper 65M Oct 31 09:09 log.997fbb
  5.5M -rw-r--r-- 1 zookeeper zookeeper 5.5M Oct 20 13:10 snapshot.96b52e
  5.5M -rw-r--r-- 1 zookeeper zookeeper 5.5M Oct 20 13:11 snapshot.96b55a
  5.5M -rw-r--r-- 1 zookeeper zookeeper 5.5M Oct 20 13:36 snapshot.96b567
  5.5M -rw-r--r-- 1 zookeeper zookeeper 5.5M Oct 24 16:17 snapshot.981572
  5.6M -rw-r--r-- 1 zookeeper zookeeper 5.6M Oct 28 21:44 snapshot.997fb9
  

• zookeeper.jar and slf4j-api.jar in /usr/share/java

Although it adds about 66Mb each time helios is run, it's worth running only when a helios deployment has been successful and, even then, keep some history. We have a convention that at least 5 "steps" should be kept.

You might want to script this in a nice command (which you'd need to run with sudo):

#!/bin/sh

cd /usr/share/java

java -cp zookeeper.jar:log4j.jar:slf4j-api.jar:conf \
    org.apache.zookeeper.server.PurgeTxnLog \
    /var/lib/zookeeper \
    -n 5

cd -

HTH,