Kafka: Deploy Kafka as Container: Difference between revisions
(Created page with "= Deploy Kafka as Container = In Linux, create a directory # Create dirs for Kafka / ZK data. mkdir -p /kafka-vol/zk-data mkdir -p /kafka-vol/zk-txn-logs mkdir -p /kafka-vol/kafka-data # Make sure the user has the read and write permissions. chown -R 1000:1000 /kafka-vol/zk-data chown -R 1000:1000 /kafka-vol/zk-txn-logs chown -R 1000:1000 /kafka-vol/kafka-data Deploy Zookeper container using Podman podman run -d --name=sb2b-zookeeper -p 32181:32181 \ -...") |
|||
| Line 12: | Line 12: | ||
chown -R 1000:1000 /kafka-vol/zk-txn-logs | chown -R 1000:1000 /kafka-vol/zk-txn-logs | ||
chown -R 1000:1000 /kafka-vol/kafka-data | chown -R 1000:1000 /kafka-vol/kafka-data | ||
Pull images | |||
podman pull docker.io/confluentinc/cp-zookeeper:latest | |||
podman pull docker.io/confluentinc/cp-kafka:latest | |||
podman pull quay.io/cloudhut/kowl:master | |||
Deploy Zookeper container using Podman | Deploy Zookeper container using Podman | ||
| Line 38: | Line 45: | ||
Now open a browser and access the url http://<your server>:8080/ | Now open a browser and access the url http://<your server>:8080/ | ||
= Configure Containers as Linux Service = | = Configure Containers as Linux Service = | ||
Revision as of 16:22, 29 October 2023
Deploy Kafka as Container
In Linux, create a directory
# Create dirs for Kafka / ZK data. mkdir -p /kafka-vol/zk-data mkdir -p /kafka-vol/zk-txn-logs mkdir -p /kafka-vol/kafka-data # Make sure the user has the read and write permissions. chown -R 1000:1000 /kafka-vol/zk-data chown -R 1000:1000 /kafka-vol/zk-txn-logs chown -R 1000:1000 /kafka-vol/kafka-data
Pull images
podman pull docker.io/confluentinc/cp-zookeeper:latest podman pull docker.io/confluentinc/cp-kafka:latest podman pull quay.io/cloudhut/kowl:master
Deploy Zookeper container using Podman
podman run -d --name=sb2b-zookeeper -p 32181:32181 \ -e ZOOKEEPER_CLIENT_PORT=32181 \ -e ZOOKEEPER_TICK_TIME=2000 \ -v /kafka-vol/zk-data:/var/lib/zookeeper/data \ -v /kafka-vol/zk-txn-logs:/var/lib/zookeeper/log \ confluentinc/cp-zookeeper:latest
Deploy Kafka container using Podman
podman run -d --name=sb2b-kafka --user=1000 --net=host \ -e KAFKA_BROKER_ID=1 \ -e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 \ -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \ -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ -v /kafka-vol/kafka-data:/var/lib/kafka/data \ confluentinc/cp-kafka:latest
Deploy Kwol container front-end using Podman
podman run --net=host --name=sb2b-kown -d -e KAFKA_BROKERS=localhost:29092 quay.io/cloudhut/kowl:master
Now open a browser and access the url http://<your server>:8080/
Configure Containers as Linux Service
Configure Zookeeper Service
Create file /etc/systemd/system/zookeeper-container.service, and add the following content
[Unit] Description=Zookeeper container [Service] Restart=always ExecStart=/usr/bin/podman start -a sb2b-zookeeper ExecStop=/usr/bin/podman stop -t 2s sb2b-zookeeper [Install] WantedBy=local.target
save the file
Enable the service
systemctl enable zookeeper-container.service
Start the service
systemctl start zookeeper-container.service
Check the status of the service
systemctl status zookeeper-container.service
Configure Kafka Service
Create file /etc/systemd/system/kafka-container.service, and add the following content
[Unit] Description=Kafka container [Service] Restart=always ExecStart=/usr/bin/podman start -a sb2b-kafka ExecStop=/usr/bin/podman stop -t 2s sb2b-kafka [Install] WantedBy=local.target
save the file
Enable the service
systemctl enable kafka-container.service
Start the service
systemctl start kafka-container.service
Check the status of the service
systemctl status kafka-container.service