Install and run MongoDB with Homebrew

• The first step is updating Homebrew. Open the Terminal application and type: brew update
• After updating Homebrew: brew install mongodb
• After downloading Mongo, create the db directory. This is where the Mongo data files will be stored: mkdir-p /data/db
• Make sure that the /data/db directory has the right permissions by running: sudo chown -R `id -un` /data/db

Creating the Replica Set

1. In different terminals, we can start three nodes for the Replica by running:

mongod --port 27017 --dbpath /data/db/rs0-0 --replSet rs0 --bind_ip localhost

mongod --port 27018 --dbpath /data/db/rs0-1 --replSet rs0 --bind_ip localhost

mongod --port 27019 --dbpath /data/db/rs0-2 --replSet rs0 --bind_ip localhost

2. For the next step, we need to open a mongo shell that connects to the primary node by using the mongo command and run:

rs.initiate(
    {
        _id: "rs0",
        members: [
            { _id: 0, host : "127.0.0.1:27017" },
            { _id: 1, host : "127.0.0.1:27018" },
            { _id: 2, host : "127.0.0.1:27019" }
        ]
    }
) 

Troubleshooting

Sometimes, one of the common MongoDB ports might be already in use. In order to find the process that is running on a specific ports, we can run:

lsof -n -i :<port-number> | grep LISTEN

This will provide us the process id, which we can kill later by running:

kill -9 <process-id>

1) Start Zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties

2) Start the Apache Kafka Server

bin/kafka-server-start.sh config/server.properties

3) Create a test topic

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

4) List the topics

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

5) Start a producer to send messages

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

6) Start a consumer to dump out the messages

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

7) Send messages from the producer and check the output in the consumer

1) In the AWS console, navigate to IAM.
2) Find your user and generate the Access Key and Secret Access Key
3) Run the following commands in terminal to paste the keys in /.aws/credentials so AWS recognizes them:

mkdir ~/.aws
vi ~/.aws/credentials
[default]
aws_access_key_id = ********************
aws_secret_access_key = ********************


4) Save and exit