Kafka（一）在WSL单机搭建Kafka伪集群

[1 运行Kafka单实例](#1 运行Kafka单实例)
- [1.1 Windws](#1.1 Windws)
- - [1.1.1 安装包下载](#1.1.1 安装包下载)
  - [1.1.2 修改环境变量](#1.1.2 修改环境变量)
  - [1.1.3 修改配置文件](#1.1.3 修改配置文件)
  - [1.1.4 启动Kafka单机版](#1.1.4 启动Kafka单机版)
- [1.2 Linux](#1.2 Linux)
- - [1.2.1 安装包下载](#1.2.1 安装包下载)
  - [1.2.2 创建目录](#1.2.2 创建目录)
  - [1.2.3 添加环境变量](#1.2.3 添加环境变量)
  - [1.2.4 修改配置文件](#1.2.4 修改配置文件)
  - [1.2.5 运行Kafka](#1.2.5 运行Kafka)
  - [1.2.6 停止Kafka](#1.2.6 停止Kafka)
- [2 搭建Kafka集群](#2 搭建Kafka集群)
- - [2.1 搭建Zookeeper集群](#2.1 搭建Zookeeper集群)
  - [2.2 搭建Kafka集群](#2.2 搭建Kafka集群)
  - - [2.2.1 新建集群目录](#2.2.1 新建集群目录)
    - [2.2.2 配置环境变量](#2.2.2 配置环境变量)
    - [2.2.3 创建节点目录](#2.2.3 创建节点目录)
    - [2.2.4 修改配置](#2.2.4 修改配置)
    - [2.2.5 启动集群](#2.2.5 启动集群)
    - [2.2.6 停止集群](#2.2.6 停止集群)

1 运行Kafka单实例

1.1 Windws

环境：Windows 11

1.1.1 安装包下载

官网下载地址：https://kafka.apache.org/downloads，截至此文，最新版本是3.5.1。

压缩包解压至本地目录，例如：D:\00_programming\kafka_2.12-3.5.1。

1.1.2 修改环境变量

KAFKA_HOME=D:\00_programming\kafka_2.12-3.5.1

修改Path变量：%KAFKA_HOME%\bin

1.1.3 修改配置文件

新建目录：D:\00_programming\kafka_2.12-3.5.1\kafka-logs

修改Kafka配置文件：D:\00_programming\kafka_2.12-3.5.1\config\server.properties

log.dirs=/00_programming/kafka_2.12-3.5.1/kafka-logs

修改Zookeeper配置文件：D:\00_programming\kafka_2.12-3.5.1\config\zookeeper.properties

dataDir=/00_programming/kafka_2.12-3.5.1/zookeeper

1.1.4 启动Kafka单机版

依次运行如下命令，每次新打开一个命令行窗口：

启动Zookeeper：

zookeeper-server-start.bat %KAFKA_HOME%\config\zookeeper.properties

启动Kafka：

kafka-server-start.bat %KAFKA_HOME%\config\server.properties

新建Topic：

kafka-topics.bat --create --bootstrap-server localhost:9092 --topic test --partitions 1 --replication-factor 1

启动生产者：

kafka-console-producer.bat --broker-list localhost:9092 --topic test

启动消费者：

kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test

简单测试：

在生产者命令行输入：

Hello, Kafka!

在消费者控制台收到消息：

Hello, Kafka!

1.2 Linux

环境：Windows 11 WSL2

Linux发行版本：Ubuntu 22.04.2 LTS

1.2.1 安装包下载

下载安装包

bash 复制代码

wget https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz

解压

bash 复制代码

tar xvf kafka_2.13-3.6.0.tgz -C /usr/local/bin

1.2.2 创建目录

创建数据和日志目录：

bash 复制代码

sudo mkdir $KAFKA_HOME/kafka-logs
sudo chmod -R o+w $KAFKA_HOME/kafka-logs

sudo mkdir $KAFKA_HOME/logs
sudo chmod -R o+w $KAFKA_HOME/logs

1.2.3 添加环境变量

bash 复制代码

vim ~/.bashrc

添加以下内容：

export KAFKA_HOME=/usr/local/bin/kafka_2.13-3.6.0

export PAHT= $PAHT:$ {KAFKA_HOME}/bin

加载环境变量：

bash 复制代码

source ~/.bashrc

1.2.4 修改配置文件

首先查看本机IP：

bash 复制代码

ip addr

修改配置：

bash 复制代码

vim $KAFKA_HOME/config/server.properties

修改以下内容：

listeners=PLAINTEXT://172.26.143.96:9092

advertised.listeners=PLAINTEXT://172.26.143.96:9092

log.dirs=/usr/local/bin/kafka_2.13-3.6.0/kafka-logs

1.2.5 运行Kafka

bash 复制代码

zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties
kafka-server-start.sh $KAFKA_HOME/config/server.properties

1.2.6 停止Kafka

bash 复制代码

kafka-server-stop.sh
zookeeper-server-stop.sh

2 搭建Kafka集群

2.1 搭建Zookeeper集群

过程参考：
在WSL单机搭建Zookeeper伪集群

2.2 搭建Kafka集群

2.2.1 新建集群目录

创建集群目录：

bash 复制代码

cd /usr/local/bin
sudo mkdir kafka-cluster

2.2.2 配置环境变量

bash 复制代码

vim ~/.bashrc

添加：

export KAFKA_CLUSTER_HOME=/usr/local/bin/kafka-cluster

bash 复制代码

source ~/bashrc

2.2.3 创建节点目录

将Kafka安装目录复制3份：

bash 复制代码

sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-1
sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-2
sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-3

清理之前单实例产生的数据：

bash 复制代码

sudo rm -rf kafka-cluster/kafka-1/kafka-logs
sudo rm -rf kafka-cluster/kafka-2/kafka-logs
sudo rm -rf kafka-cluster/kafka-3/kafka-logs

sudo mkdir kafka-cluster/kafka-1/kafka-logs
sudo mkdir kafka-cluster/kafka-2/kafka-logs
sudo mkdir kafka-cluster/kafka-3/kafka-logs

sudo chmod -R 777 kafka-cluster/kafka-1/kafka-logs
sudo chmod -R 777 kafka-cluster/kafka-2/kafka-logs
sudo chmod -R 777 kafka-cluster/kafka-3/kafka-logs

sudo rm -rf kafka-cluster/kafka-1/logs
sudo rm -rf kafka-cluster/kafka-2/logs
sudo rm -rf kafka-cluster/kafka-3/logs

sudo mkdir kafka-cluster/kafka-1/logs
sudo mkdir kafka-cluster/kafka-2/logs
sudo mkdir kafka-cluster/kafka-3/logs

sudo chmod -R 777 kafka-cluster/kafka-1/logs
sudo chmod -R 777 kafka-cluster/kafka-2/logs
sudo chmod -R 777 kafka-cluster/kafka-3/logs

2.2.4 修改配置

分别修改3个节点的配置

bash 复制代码

sudo vim $KAFKA_CLUSTER_HOME/kafka-1/config/server.properties

broker.id=0

listeners=PLAINTEXT://172.26.143.96:9092

advertised.listeners=PLAINTEXT://172.26.143.96:9092

log.dirs=$KAFKA_CLUSTER_HOME/kafka-1/kafka-logs

zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183

bash 复制代码

sudo vim $KAFKA_CLUSTER_HOME/kafka-2/config/server.properties

broker.id=1

listeners=PLAINTEXT://172.26.143.96:9093

advertised.listeners=PLAINTEXT://172.26.143.96:9093

log.dirs=$KAFKA_CLUSTER_HOME/kafka-2/kafka-logs

zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183

bash 复制代码

sudo vim $KAFKA_CLUSTER_HOME/kafka-3/config/server.properties

broker.id=1

listeners=PLAINTEXT://172.26.143.96:9094

advertised.listeners=PLAINTEXT://172.26.143.96:9094

log.dirs=$KAFKA_CLUSTER_HOME/kafka-3/kafka-logs

zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183

2.2.5 启动集群

bash 复制代码

$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-1/config/server.properties
$KAFKA_CLUSTER_HOME/kafka-2/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-2/config/server.properties
$KAFKA_CLUSTER_HOME/kafka-3/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-3/config/server.properties

创建Topic并查看：

bash 复制代码

$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-topics.sh --create --bootstrap-server 172.26.143.96:9092 --topic test --partitions 10 --replication-factor 3
$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-topics.sh --bootstrap-server 172.26.143.96:9092 --describe --topic test

Topic: test TopicId: yhX-1uT2SAaYj-HrreKEPw PartitionCount: 10 ReplicationFactor: 3 Configs:

Topic: test Partition: 0 Leader: 0 Replicas: 0,2,1 Isr: 0,2,1

Topic: test Partition: 1 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0

Topic: test Partition: 2 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2

Topic: test Partition: 3 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2

Topic: test Partition: 4 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1

Topic: test Partition: 5 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0

Topic: test Partition: 6 Leader: 0 Replicas: 0,2,1 Isr: 0,2,1

Topic: test Partition: 7 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0

Topic: test Partition: 8 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2

Topic: test Partition: 9 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2

2.2.6 停止集群

bash 复制代码

$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-server-stop.sh
$KAFKA_CLUSTER_HOME/kafka-2/bin/kafka-server-stop.sh
$KAFKA_CLUSTER_HOME/kafka-3/bin/kafka-server-stop.sh