Kafka Dump

Luke Hebert

- 2 minutes read - 247 words

Kafka topic dumps are one of the most common troubleshooting tasks when something looks off in a streaming pipeline. This guide walks through two approaches: a quick generic consumer dump and a full Atlas-specific workflow that handles JAAS, Kerberos, and log extraction in one pass.

Generic Kafka Dump

See the Atlas section below for the config.properties and JAAS configuration details.

  1. kafka-console-consumer --bootstrap-server <brokerHost>:<brokerPort> --topic <topicName> --from-beginning --consumer.config ~/file.properties
  2. kafka-run-class kafka.tools.DumpLogSegments --deep-iteration --print-data-log --files <log.dirs/topicPartitionFolder/file.log>

Kafka Dump for Atlas

Export the Hive Process Directory

Note

The path may vary depending on whether you are running Hive or Hive on Tez. The goal is to locate atlas-application.properties and extract the bootstrap server value. The method below populates the HIVE_PROCESS_DIR variable for Hive and then greps for the bootstrap server. You can also simply hard-code an appropriate server name or hostname for the bootstrap server.

export HIVE_PROCESS_DIR=$(ls -1dtr /var/run/cloudera-scm-agent/process/*hive_on_tez-HIVESERVER2 | tail -1)

Create Kafka Client JAAS and Config File

Prepare the SASL and Kerberos configuration files needed to authenticate against the Kafka broker.

echo 'security.protocol=SASL_PLAINTEXT' > /tmp/kafka.config
cat > /tmp/kafka_jaas.conf <<'EOF'
KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
        useTicketCache=true
        renewTicket=true
        serviceName="kafka";
};
EOF

Obtain Kerberos Credentials

On public cloud, use an appropriate workload user with kinit and a keytab:

kinit -kt /path/to/workload.keytab workloaduser@REALM

Verify the ticket is active before proceeding:

klist

Obtain the Dump

export KAFKA_OPTS='-Djava.security.auth.login.config=/tmp/kafka_jaas.conf'
export BOOTSTRAP_SERVER=$(cat $HIVE_PROCESS_DIR/atlas-application.properties | grep bootstrap | awk -F'=' '{print $2}')

kafka-console-consumer --bootstrap-server ${BOOTSTRAP_SERVER} --topic ATLAS_HOOK --command-config /tmp/kafka.config --from-beginning > /tmp/atlas_hook.log