docker-compose部署 kafka 3.7 集群(3台服务器)并启用账号密码认证

文章目录

  • [1. 规划](#1. 规划)
  • [2. 服务部署](#2. 服务部署)
    • [2.1 kafka-01](#2.1 kafka-01)
    • [2.2 kafka-02](#2.2 kafka-02)
    • [2.3 kafka-03](#2.3 kafka-03)
    • [2.4 启动服务](#2.4 启动服务)
  • [3. 测试](#3. 测试)
    • [3.1 kafkamap搭建(测试工具)](#3.1 kafkamap搭建(测试工具))
    • [3.2 测试](#3.2 测试)

1. 规划

服务 IP
kafka-01 10.10.xxx.199
kafka-02 10.10.xxx.198
kafka-03 10.10.xxx.197
kafkamp 10.10.xxx.199

2. 服务部署

2.1 kafka-01

  • 登录10.10.xxx.199,创建目录 kafka-01,进入该目录
  • 目录下创建docker-compose.yml 文件
yml 复制代码
version: "3"
services:
   kafka:
     image: 'bitnami/kafka:3.7.0'
     user: root
     ports:
       - '9092:9092'
       - '9093:9093'
     environment:
       - TZ=Asia/Shanghai
       - KAFKA_CFG_PROCESS_ROLES=broker,controller
       - BITNAMI_DEBUG=true 
       - ALLOW_PLAINTEXT_LISTENER=no
       - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
       - KAFKA_CFG_NUM_PARTITIONS=1
       - KAFKA_CFG_LISTENERS=INTERNAL://:9094,CLIENT://:9095,CONTROLLER://0.0.0.0:9093,EXTERNAL://0.0.0.0:9092
       - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://10.10.181.199:9094,CLIENT://:9095,EXTERNAL://10.10.181.199:9092
       - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:SASL_PLAINTEXT,CLIENT:SASL_PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT   
       - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
       - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
       - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
       - KAFKA_CLIENT_USERS=liubei
       - KAFKA_CLIENT_PASSWORDS=liubei@161
       - KAFKA_INTER_BROKER_USER=liuwei
       - KAFKA_INTER_BROKER_PASSWORD=liubei@161
       - KAFKA_CFG_NODE_ID=1
       - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@10.10.181.199:9093,2@10.10.181.198:9093,3@10.10.181.197:9093
       - KAFKA_KRAFT_CLUSTER_ID=xishu
       - KAFKA_OPTS=-Djava.security.auth.login.config=/bitnami/kafka/data/kafka_server_jaas.conf
     volumes:
       - ./data:/bitnami/kafka/data:rw

2.2 kafka-02

  • 登录10.10.xxx.198,创建目录 kafka-02,进入该目录
  • 目录下创建docker-compose.yml 文件
yml 复制代码
version: "3"
services:
   kafka:
     image: bitnami/kafka:3.7.0'
     user: root
     ports:
       - '9092:9092'
       - '9093:9093'
     environment:
       - TZ=Asia/Shanghai
       - KAFKA_CFG_PROCESS_ROLES=broker,controller
       - BITNAMI_DEBUG=true 
       - ALLOW_PLAINTEXT_LISTENER=no
       - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
       - KAFKA_CFG_NUM_PARTITIONS=1
       - KAFKA_CFG_LISTENERS=INTERNAL://:9094,CLIENT://:9095,CONTROLLER://0.0.0.0:9093,EXTERNAL://0.0.0.0:9092
       - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://10.10.181.198:9094,CLIENT://:9095,EXTERNAL://10.10.181.198:9092
       - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:SASL_PLAINTEXT,CLIENT:SASL_PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT   
       - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
       - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
       - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
       - KAFKA_CLIENT_USERS=liubei
       - KAFKA_CLIENT_PASSWORDS=liubei@161
       - KAFKA_INTER_BROKER_USER=liuwei
       - KAFKA_INTER_BROKER_PASSWORD=liubei@161
       - KAFKA_CFG_NODE_ID=2
       - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@10.10.181.199:9093,2@10.10.181.198:9093,3@10.10.181.197:9093
       - KAFKA_KRAFT_CLUSTER_ID=xishu
       - KAFKA_OPTS=-Djava.security.auth.login.config=/bitnami/kafka/data/kafka_server_jaas.conf
     volumes:
       - ./data:/bitnami/kafka/data:rw

2.3 kafka-03

  • 登录10.10.xxx.197,创建目录 kafka-03,进入该目录
  • 目录下创建docker-compose.yml 文件
yml 复制代码
version: "3"
services:
   kafka:
     image: bitnami/kafka:3.7.0'
     user: root
     ports:
       - '9092:9092'
       - '9093:9093'
     environment:
       - TZ=Asia/Shanghai
       - KAFKA_CFG_PROCESS_ROLES=broker,controller
       - BITNAMI_DEBUG=true 
       - ALLOW_PLAINTEXT_LISTENER=no
       - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
       - KAFKA_CFG_NUM_PARTITIONS=1
       - KAFKA_CFG_LISTENERS=INTERNAL://:9094,CLIENT://:9095,CONTROLLER://0.0.0.0:9093,EXTERNAL://0.0.0.0:9092
       - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://10.10.181.197:9094,CLIENT://:9095,EXTERNAL://10.10.181.197:9092
       - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:SASL_PLAINTEXT,CLIENT:SASL_PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT   
       - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
       - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
       - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
       - KAFKA_CLIENT_USERS=liubei
       - KAFKA_CLIENT_PASSWORDS=liubei@161
       - KAFKA_INTER_BROKER_USER=liuwei
       - KAFKA_INTER_BROKER_PASSWORD=liubei@161
       - KAFKA_CFG_NODE_ID=2
       - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@10.10.181.199:9093,2@10.10.181.198:9093,3@10.10.181.197:9093
       - KAFKA_KRAFT_CLUSTER_ID=xishu
       - KAFKA_OPTS=-Djava.security.auth.login.config=/bitnami/kafka/data/kafka_server_jaas.conf
     volumes:
       - ./data:/bitnami/kafka/data:rw

2.4 启动服务

  • 每台服务器上执行
shell 复制代码
docker-compose up -d

3. 测试

3.1 kafkamap搭建(测试工具)

  • 登录10.10.xxx.199,创建kafkamap目录,进入该目录
  • 创建docker-compose.yml文件如下
shell 复制代码
version: "3"
services:
   kafka_map:
     image: dushixiang/kafka-map:latest
     environment:
       - DEFAULT_USERNAME=liubei
       - DEFAULT_PASSWORD=liubei@161
     ports:
       - '8080:8080'

3.2 测试

  • 连接集群
  • 连接成功
  • 进入topic
  • 创建topic

相关推荐
布莱克6056 小时前
Swift 类详解:从定义、继承到内存管理的完整指南
服务器·开发语言·ios·ssh·swift·类
布莱克6058 小时前
Swift 方法详解:实例方法、类型方法与 mutating 机制
服务器·开发语言·ios·ssh·swift·方法
Rabitebla9 小时前
【Linux】文件篇(五):手撕简易 FILE——模拟实现 C 标准 I/O 缓冲机制
linux·服务器·c语言·笔记·学习·ubuntu
zly350015 小时前
esxi安装 windows2000 鼠标不动
运维·服务器
LabVIEW开发16 小时前
Ethernet/IP 会话无效 -251723770
服务器·网络协议·tcp/ip
AR-26710-18 小时前
docker Day2——镜像 / 容器 / 仓库
运维·docker·容器
Huangjin007_19 小时前
【Linux 系统篇(四十一)】线程(二):线程的优缺点、资源共享与上下文切换
linux·运维·服务器
好用斋19 小时前
拾光坞G2使用新版Docker懒人包:一键安装、一键升级、一键卸载
java·docker·容器
Rabitebla20 小时前
【Linux】文件篇(六):Shell 实战——从系统调用到命令行重定向
linux·服务器·c++·笔记·学习
csdn_aspnet20 小时前
生产环境中 SQL Server 的 Docker 使用建议
运维·microsoft·docker·容器·sqlserver