Kafka command-line tool

Posted on by 솔천

reference: https://kafka.apache.org/quickstart

Download Kafka

download the Kafka client at https://www.apache.org/dyn/closer.cgi?path=/kafka/2.1.0/kafka_2.11-2.1.0.tgz and unzip the folder

cd kafka_2.11-2.1.0/bin

Run Kafka commands

Topics

All topics

$ ./kafka-topics.sh --bootstrap-server kafka-name.hostname.com:9092 --list
__consumer_offsets
_schemas
service
   service.database.table
...

topic detail

$ ./kafka-topics.sh --describe --bootstrap-server kafka-name.hostname.com:9092 --topic topicname
Topic:botr.botr.media PartitionCount:1 ReplicationFactor:3 Configs:min.compaction.lag.ms=604800000,min.insync.replicas=2,min.cleanable.dirty.ratio=0.05,delete.retention.ms=604800000,cleanup.policy=compact
Topic: botr.botr.media Partition: 0 Leader: 2 Replicas: 2,1,0 Isr: 1,0,2

Consumers

list of consumers

$ ./kafka-consumer-groups.sh --bootstrap-server kafka-name.hostname.com:9092 --list
consumer-0
consumer-1
consumer-2
...

Show consumer detail

./kafka-consumer-groups.sh --bootstrap-server kafka-name.hostname.com:9092 --describe --group kafka-group-name

Messages

Show messages from topic

./kafka-console-consumer.sh --bootstrap-server kafka-name.hostname.com:9092 --topic kafka.topic.name

show all messages

./kafka-console-consumer.sh --bootstrap-server kafka-name.hostname.com:9092 --topic kafka.topic.name --from-beginning