Docker部署kafka集群

1,编写Docker Compose文件

编写一个docker-compose.yml文件来定义服务,以下定义了一个Zookeeper服务和三个Kafka Broker服务:

注意:把10.0.8.4替换成宿主IP

java 复制代码
version: '3.8'

services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    container_name: ncc-zookeeper
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    networks:
      - kafka-net

  kafka1:
    image: 'bitnami/kafka:latest'
    container_name: ncc-kafka1
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ADVERTISED_HOST_NAME=10.0.8.4
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://10.0.8.4:9092
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    depends_on:
      - zookeeper
    networks:
      - kafka-net

  kafka2:
    image: 'bitnami/kafka:latest'
    container_name: ncc-kafka2
    ports:
      - '9093:9092'
    environment:
      - KAFKA_BROKER_ID=2
      - KAFKA_ADVERTISED_HOST_NAME=10.0.8.4
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://10.0.8.4:9093
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    depends_on:
      - zookeeper
    networks:
      - kafka-net

  kafka3:
    image: 'bitnami/kafka:latest'
    container_name: ncc-kafka3
    ports:
      - '9094:9092'
    environment:
      - KAFKA_BROKER_ID=3
      - KAFKA_ADVERTISED_HOST_NAME=10.0.8.4
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://10.0.8.4:9094
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    depends_on:
      - zookeeper
    networks:
      - kafka-net

networks:
  kafka-net:
    driver: bridge
    external: false

2,启动Kafka集群

bash 复制代码
sudo docker-compose up -d

验证安装

一旦所有服务都已启动并运行,你可以通过连接到任意一个Kafka Broker来验证安装是否成功。例如,可以使用Kafka自带的命令行工具创建一个测试主题:

3,在ncc-kafka1创建主题,然后在ncc-kafka2查看主题列表

bash 复制代码
sudo docker exec -it ncc-kafka1 /opt/bitnami/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server ncc-kafka1:9092 --replication-factor 3 --partitions 1
bash 复制代码
sudo docker exec -it ncc-kafka2 /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server ncc-kafka2:9092

4,验证发送消息

bash 复制代码
sudo docker exec -it ncc-kafka1 /opt/bitnami/kafka/bin/kafka-console-producer.sh --broker-list ncc-kafka1:9092 --topic test-topic

另外新开一个窗口,执行

bash 复制代码
sudo docker exec -it ncc-kafka3 /opt/bitnami/kafka/bin/kafka-console-consumer.sh --bootstrap-server ncc-kafka3:9092 --topic test-topic --from-beginning

测试验证成功

相关推荐
有谁看见我的剑了?4 小时前
k8s storageclasses nfs-provisioner 部署
docker·容器·kubernetes
故事很腻i4 小时前
安装kafka-2.5.1
分布式·kafka
忧郁的橙子.5 小时前
安装插件遇到报错时间错误导致签证失败 Errors during downloading metadata for repository ‘baseos‘:
docker
坐吃山猪5 小时前
Kafka11-知识补充
kafka
超龄超能程序猿6 小时前
Docker 镜像的导出和还原操作
运维·docker·容器
维尔切6 小时前
Docker 监控平台部署
运维·docker·容器·grafana·prometheus
温柔一只鬼.6 小时前
Docker快速入门——第三章Docker环境安装
运维·docker·容器
红尘客栈27 小时前
k8s的ymal文件
docker·容器·kubernetes
hello_2507 小时前
k8s opa集成
云原生·容器·kubernetes
yumgpkpm7 小时前
CMP平台(类Cloudera CDP7.3)在华为鲲鹏的Aarch64信创环境中的性能表现
大数据·flink·kafka·big data·flume·cloudera