在k8s上使用strimzi operator安装kafka集群

文章目录

• • 部署步骤

Strimzi Operator 是一种强大的工具，它简化了 Kafka 在 Kubernetes 环境中的部署和管理, 记录下部署教程

部署步骤

官方地址: https://strimzi.io/quickstarts/

创建命名空间:

kubectl create namespace kafka-system

1. 安装operator, 完成后

   kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka-system' -n kafka-system

   如果k8s集群域名不是默认的cluster.local, 需要在operator资源上使用KUBERNETES_SERVICE_DNS_DOMAIN环境变量覆盖原来的

2. 执行kubectl apply -f kafka.yaml -n kafka-system

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaNodePool
   metadata:
   name: dual-role
   labels:
   strimzi.io/cluster: my-cluster
   spec:
   replicas: 3
   roles:
   - controller
   - broker
   resources:
   requests:
   memory: 4Gi
   cpu: "2"
   limits:
   memory: 4Gi
   cpu: "2"
   storage:
   type: jbod
   volumes:
   - id: 0
   type: persistent-claim
   size: 20Gi
   class: disk-ssd
   deleteClaim: false
   kraftMetadata: shared

   apiVersion: kafka.strimzi.io/v1beta2
   kind: Kafka
   metadata:
   name: my-cluster
   annotations:
   strimzi.io/node-pools: enabled
   strimzi.io/kraft: enabled
   spec:
   kafka:
   version: 3.8.0
   metadataVersion: 3.8-IV0
   listeners:
   - name: plain
   port: 9092
   type: internal
   tls: false
   - name: tls
   port: 9093
   type: internal
   tls: true
   config:
   offsets.topic.replication.factor: 3
   transaction.state.log.replication.factor: 3
   transaction.state.log.min.isr: 2
   default.replication.factor: 3
   min.insync.replicas: 2
   entityOperator:
   topicOperator: {}
   userOperator: {}

3. 创建生产者:
   kubectl -n kafka-system run kafka-producer -ti --image=quay.io/strimzi/kafka:0.45.0-kafka-3.8.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic

4. 创建消费者:
   kubectl -n kafka-system run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.45.0-kafka-3.8.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --from-beginning

其他示例:

https://github.com/strimzi/strimzi-kafka-operator/blob/release-0.43.x/examples/kafka/kafka-persistent.yaml

说明:

kafka-ephemeral-single.yaml：非持久化存储，单节点集群；

kafka-ephemeral.yaml：非持久化存储，多节点集群；

kafka-jbod.yaml：jbod存储，多节点集群；

kafka-persistent-single.yaml：持久化存储，单节点集群；

kafka-persistent.yaml ：持久化存储，多节点集群；