Kafka CLI

Once working with Kafka, I often need to quickly interact with Kafka cluster via command line. This post is my collection of frequent commands used in daily work with projects having Kafka integrated.

Listing topics

• The --list switch tells the kafka-topics CLI to list all known topics.
• The --zookeeper localhost:2181 switch tells the kafka-topics CLI where the Zookeeper ensemble Kafka is using is located. Note that in the newest versions of Kafka the --zookeeper switch is deprecated in favor of a --bootstrap-server switch that points to Kafka. The --zookeeper switch still works, but will likely be dropped in the next major revision of Kafka.

Creating topics

• The switch --topic "my-first-topic" tells Kafka what to name the topic
• The switch --partitions 1 and the switch --replication-factor 1 are required configuration
• To check that our topic was successfully created, let’s repeat the command to list topics with a slight modification:

Producing data

• The switch --broker-list serves the same purpose as --zookeeper for the kafka-topics command, it simply tells the tool where to find Kafka.
• Try typing out a few messages and hitting enter.

Consuming data

• The --from-beginning switch tells kafka-console-consumer to read data from the beginning of the topic, not just data from when we connect.
• By default Kafka doesn’t provide historical messages to new consumers
• You should now see output that includes all of the messages you’ve produced:

Deleting topics

• This command is nearly identical to the –create command from earlier, except now we’re calling the command with the –delete switch instead.
• This command does not print any output if it’s successful. To check that your topic is actually deleted, list the topics one more time:

Describe topic

• This is helpful once there is a need of understanding how data is managed by Kafka.