对已运行的kafka集群添加副本

在已经运行的kafka集群中(业务主题已创建),如果一开始配置的默认副本是1,则后续再去修改配置文件为3,也只是针对后续新增的主题有效。原先主题不会生效。需要通过json文件去修改。

先创建__consumer_offsets主题的JSON

注意json文件中的"replicas":[2,1,0]这些是集群中的broke,有可能有的broke是1,2,3

新建一个consumer_offsets.json的文件,内容如下

{
"version": 1,
"partitions": [
{"topic": "__consumer_offsets","partition": 0,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 1,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 2,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 3,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 4,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 5,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 6,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 7,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 8,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 9,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 10,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 11,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 12,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 13,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 14,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 15,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 16,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 17,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 18,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 19,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 20,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 21,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 22,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 23,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 24,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 25,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 26,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 27,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 28,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 29,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 30,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 31,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 32,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 33,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 34,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 35,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 36,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 37,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 38,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 38,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 39,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 40,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 41,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 42,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 43,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 44,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 45,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 46,"replicas": [1,0]},
{"topic": "__consumer_offsets","partition": 47,"replicas": [2,1]},
{"topic": "__consumer_offsets","partition": 48,"replicas": [0,2]},
{"topic": "__consumer_offsets","partition": 49,"replicas": [1,0]}
]
}

在任意一个节点上执行(只需在一个节点上执行一次)

bash 复制代码
bin/kafka-reassign-partitions.sh --zookeeper 192.168.1.1:2181 --reassignment-json-file consumer_offsets.json --execute

之后再加业务主题副本,新建一个doorinout.json的文件,内容如下

bash 复制代码
{
"version": 1,
"partitions": [
{"topic": "doorinout","partition": 0,"replicas": [2,1]},
{"topic": "doorinout","partition": 1,"replicas": [0,2]},
{"topic": "doorinout","partition": 2,"replicas": [1,0]},
]
}

在任意一个节点上执行(只需在一个节点上执行一次)

bash 复制代码
bin/kafka-reassign-partitions.sh --zookeeper 192.168.1.1:2181 --reassignment-json-file doorinout.json --execute
相关推荐
龙哥·三年风水1 分钟前
群控系统服务端开发模式-应用开发-个人资料
分布式·php·群控系统
funnyZpC2 小时前
quartz集群增强版🎉
java·分布式·开源·集群·定时任务
明达技术3 小时前
工业4.0时代下的分布式IO模块
分布式
天冬忘忧4 小时前
Spark 程序开发与提交:本地与集群模式全解析
大数据·分布式·spark
一叶飘零_sweeeet6 小时前
Dubbo 构建高效分布式服务架构
分布式·架构·dubbo
孤蓬&听雨6 小时前
RabbitMQ自动发送消息工具(自动化测试RabbitMQ)
分布式·测试工具·自动化·rabbitmq·自动发送消息
呼啦啦啦啦啦啦啦啦6 小时前
RabbitMQ 七种工作模式介绍
分布式·rabbitmq
qq_203769496 小时前
win11安装最新rabbitmq
分布式·rabbitmq·ruby
码农爱java6 小时前
Kafka 之消息并发消费
spring boot·微服务·kafka·mq·消息中间件·并发消费
Amagi.6 小时前
RabbitMQ的解耦、异步、削峰是什么?
分布式·rabbitmq