Kafka: Deploy Kafka as Container

Deploy Kafka as Container

In Linux, create a directory

# Create dirs for Kafka / ZK data.
mkdir -p /kafka-vol/zk-data
mkdir -p /kafka-vol/zk-txn-logs
mkdir -p /kafka-vol/kafka-data

# Make sure the user has the read and write permissions.
chown -R 1000:1000 /kafka-vol/zk-data
chown -R 1000:1000 /kafka-vol/zk-txn-logs
chown -R 1000:1000 /kafka-vol/kafka-data

Pull images

podman pull docker.io/confluentinc/cp-zookeeper:latest
podman pull docker.io/confluentinc/cp-kafka:latest
podman pull quay.io/cloudhut/kowl:master

Deploy Zookeper container using Podman

podman run -d --name=sb2b-zookeeper -p 32181:32181 \
   -e ZOOKEEPER_CLIENT_PORT=32181 \
   -e ZOOKEEPER_TICK_TIME=2000 \
   -v /kafka-vol/zk-data:/var/lib/zookeeper/data \
   -v /kafka-vol/zk-txn-logs:/var/lib/zookeeper/log \
   confluentinc/cp-zookeeper:latest

Deploy Kafka container using Podman

podman run -d --name=sb2b-kafka --user=1000 --net=host \
   -e KAFKA_BROKER_ID=1 \
   -e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 \
   -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \
   -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
   -v /kafka-vol/kafka-data:/var/lib/kafka/data \
   confluentinc/cp-kafka:latest

Deploy Kwol container front-end using Podman

podman run --net=host --name=sb2b-kowl -d -e KAFKA_BROKERS=localhost:29092 quay.io/cloudhut/kowl:master 

Create topics using Kowl

Now open a browser and access the url http://<your server>:8080/

Configure Containers as Linux Service

Configure Zookeeper Service

Create file /etc/systemd/system/zookeeper-container.service, and add the following content

[Unit]
Description=Zookeeper container

[Service]
Restart=always
ExecStart=/usr/bin/podman start -a sb2b-zookeeper
ExecStop=/usr/bin/podman stop -t 2s sb2b-zookeeper

[Install]
WantedBy=local.target

save the file

Enable the service

systemctl enable zookeeper-container.service

Start the service

systemctl start zookeeper-container.service

Check the status of the service

systemctl status zookeeper-container.service

Configure Kafka Service

Create file /etc/systemd/system/kafka-container.service, and add the following content

[Unit]
Description=Kafka container

[Service]
Restart=always
ExecStart=/usr/bin/podman start -a sb2b-kafka
ExecStop=/usr/bin/podman stop -t 2s sb2b-kafka

[Install]
WantedBy=local.target

save the file

Enable the service

systemctl enable kafka-container.service

Start the service

systemctl start kafka-container.service

Check the status of the service

systemctl status kafka-container.service

Ver também

• Artigos sobre Kafka