Mini-Recipes: Apache Kafka Quickstart Commands

1) Start Zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties

2) Start the Apache Kafka Server

bin/kafka-server-start.sh config/server.properties

3) Create a test topic

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

4) List the topics

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

5) Start a producer to send messages

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

6) Start a consumer to dump out the messages

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

7) Send messages from the producer and check the output in the consumer

Advertisements

Mini-Recipes: How to setup AWS credentials

1) In the AWS console, navigate to IAM.
2) Find your user and generate the Access Key and Secret Access Key
3) Run the following commands in terminal to paste the keys in /.aws/credentials so AWS recognizes them:

mkdir ~/.aws
vi ~/.aws/credentials
[default]
aws_access_key_id = ********************
aws_secret_access_key = ********************

4) Save and exit