Kafka Quick Start Guide

Installing Kafka in Linux(Ubuntu) machines. This simple step-by-step by guide elaborates how to install kafka in local linux flavored machines and do a simple testing of working flow.

----------------------------------------
1. Create kafka user in Server
----------------------------------------
1. Create linux user for Kafka setup. From root user create user as kafka
useradd kafka -m

2. Set password for user kafka
passwd kafka

3. Add the user in sudo group to give all privileges required to install Kafka binaries and it's dependencies.
adduser kafka sudo

----------------------------------------
2. Install Java
----------------------------------------

1. Update all current binaries in linux
sudo apt-get update

2. Install Java 8.0 version
sudo apt-get install default-jre

3. Verify Java version
java -version

----------------------------------------
3. Install Zookeeper
----------------------------------------

1. Install zookeeper from repository
sudo apt-get install zookeeperd

2. Verify zookeeperd
telnet localhost 2181

---------------------------------------------
3. Downloading and installing Kafka Binaries
---------------------------------------------
1. Create directory Downloads in home
mkdir -p ~/Downloads 

2. Download Kafka from Apache repository to localhost
wget "http://www-eu.apache.org/dist/kafka/0.11.0.1/kafka_2.11-0.11.0.1.tgz" -O ~/Downloads/kafka.tgz 

3. Create directory kafka and uncompress the downloaded tar file into kafka folder
mkdir -p ~/kafka && cd ~/kafka 

tar -xvzf ~/Downloads/kafka.tgz --strip 1

---------------------------------------------
4. Configuring Kafka server
---------------------------------------------

vi ~/kafka/config/server.properties

To allow the deletion of topics in Kafka server add the below line in server.properties file

delete.topic.enable = true

---------------------------------------------
5. Start the Kafka server
---------------------------------------------
nohup ~/kafka/bin/kafka-server-start.sh ~/kafka/config/server.properties > ~/kafka/kafka.log 2>&1 & 

---------------------------------------------
6. Create a topic in Kafka server
---------------------------------------------
~/kafka/bin/kafka-create-topic.sh --zookeeper localhost:2181 --replica 1 --partition 1 --topic test

~/kafka/bin/kafka-list-topic.sh --zookeeper localhost:2181

---------------------------------------------
7. Start the kafka producer console 
---------------------------------------------

Start producer console and send messages to topic test. Parallel you can start the consumer in another terminal and see real time message flow. 

~/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test 
This is a test message

---------------------------------------------
6. Start the kafka consumer console 
---------------------------------------------
Start the consumer console and see the messages from topic test displayed in terminal

~/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
This is a test message