K8S下redis哨兵集群使用secret隐藏configmap内明文密码方案详解

斯普信专业组2025-02-21 17:00

#作者：朱雷

文章目录

  • 一、背景环境及方案说明
  • [二、redis secret configmap deployment参考](#二、redis secret configmap deployment参考)
    • [2.1 创建secret-redis.yaml参考](#2.1 创建secret-redis.yaml参考)
    • [2.2 修改configmap配置参考](#2.2 修改configmap配置参考)
      • [2.2.1 哨兵节点修改（每个节点都修改）](#2.2.1 哨兵节点修改（每个节点都修改）)
      • [2.2.2 主从节点配置修改](#2.2.2 主从节点配置修改)
      • [2.2.3 使用命令行参数指定密码（本小节与上面两小节互斥）](#2.2.3 使用命令行参数指定密码（本小节与上面两小节互斥）)
    • [2.3 修改deployment配置参考](#2.3 修改deployment配置参考)
      • [2.3.1 master&slaves节点的deployment的yaml修改](#2.3.1 master&slaves节点的deployment的yaml修改)
      • [2.3.2 哨兵节点的deployment的yaml修改](#2.3.2 哨兵节点的deployment的yaml修改)
      • [2.3.4 镜像环境变量参考](#2.3.4 镜像环境变量参考)
  • 三、测试是否生效
  • [四、 注意事项](#四、 注意事项)

一、背景环境及方案说明

Redis 哨兵configmap里包含明文密码，需要处理不在configmap内显示明文密码。

1.1、环境说明

方案基于Redis-5.0.14 哨兵版本（5.x，6.x版本兼容）

方案基于redis-sentinel-exporter-5.0.8版本

方案基于容器环境变量

使用两种方案的其中任意一种均可实现

1.2、方案一：使用配置文件设置密码

参照 2.2.1 和2.2.2 修改configmap

参照 2.3.1 和2.3.2 修改deployment

1.3、方案二：使用args 的命令行传参设置密码

参照 2.2.3.1 和2.2.3.2 修改configmap

参照 2.3.3.1 和2.3.3.2 修改deployment

二、redis secret configmap deployment参考

2.1 创建secret-redis.yaml参考

复制代码 
${}内为redis密码的base64的编码，若认证和同步密码不一致分别定义即可
apiVersion: v1
data:
  password: ${aGFyYm9yMjM0NSM=}
kind: Secret
metadata:
  name: redis-auth-secret
  namespace: paas-middleware

2.2 修改configmap配置参考

2.2.1 哨兵节点修改（每个节点都修改）

复制代码 
apiVersion: v1
data:
  redis-docker-entrypoint.sh: |
    #!/bin/bash
    if [ ! -f "/redis-conf/redis.conf" ]; then
        cp /etc/redis/redis.conf /redis-conf/redis.conf
        **echo -e "sentinel auth-pass mymaster ${REDIS_PASSWORD}" >> /redis-conf/redis.conf
    fi**
    redis-sentinel /redis-conf/redis.conf $@
  redis.conf: |
    port 26379
    protected-mode no
    daemonize no
    sentinel monitor mymaster 169.169.164.253 6379 2
    sentinel down-after-milliseconds mymaster 15000
    sentinel failover-timeout mymaster 60000
    sentinel deny-scripts-reconfig yes
    sentinel parallel-syncs mymaster 2
    sentinel auth-pass mymaster somepassword   # 删除这行配置
kind: ConfigMap
metadata:
  labels:
    app: redis-base-1
    type: redis
  name: redis-base-1-sentinel-1
  namespace: paas-middleware
  每个哨兵的configmap 都修改下， 有****不带删除线的为新增行

2.2.2 主从节点配置修改

复制代码 
apiVersion: v1
data:
  redis-docker-entrypoint.sh: |
    #!/bin/bash
    if [ ! -f "/redis-conf/redis.conf" ]; then
        cp /etc/redis/redis.conf /redis-conf/redis.conf
        **echo -e "masterauth ${REDIS_MASTER_PASSWORD}" >> /redis-conf/redis.conf
        echo -e "requirepass ${REDIS_PASSWORD}" >> /redis-conf/redis.conf**
    fi
    redis-server /redis-conf/redis.conf $@
  redis.conf: |
    bind 0.0.0.0 ::
    port 6379
    daemonize no
    protected-mode no
    timeout 300
    tcp-keepalive 300
    replica-read-only yes
    replica-serve-stale-data yes
    maxclients 20000
    maxmemory 0
    maxmemory-policy noeviction
    masterauth somepassword  # 删除此行配置
    requirepass somepassword  # 删除此行配置
    rename-command FLUSHALL ""
    dir "/data/"
    pidfile "/data/redis.pid"
    logfile "/data/redis.log"
kind: ConfigMap
metadata:
  labels:
    app: redis-base-1
    type: redis
  name: redis-base-1-master
  namespace: paas-middleware
 所有主从configmap配置文件都修改， 有****不带删除线的为新增行

2.2.3 使用命令行参数指定密码（本小节与上面两小节互斥）

  1. 以下为哨兵节点configmap 修改

    apiVersion: v1
    data:
    redis-docker-entrypoint.sh: |
    #!/bin/bash
    if [ ! -f "/redis-conf/redis.conf" ]; then
    cp /etc/redis/redis.conf /redis-conf/redis.conf
    fi
    redis-sentinel /redis-conf/redis.conf $@
    redis.conf: |
    port 26379
    protected-mode no
    daemonize no
    sentinel monitor mymaster 169.169.164.253 6379 2
    sentinel down-after-milliseconds mymaster 15000
    sentinel failover-timeout mymaster 60000
    sentinel deny-scripts-reconfig yes
    sentinel parallel-syncs mymaster 2
    sentinel auth-pass mymaster somepassword # 删除这行配置
    kind: ConfigMap
    metadata:
    labels:
    app: redis-base-1
    type: redis
    name: redis-base-1-sentinel-1
    namespace: paas-middleware
    每个哨兵的configmap 都修改下， 有****不带删除线的为新增行

  2. 以下为主从节点configmap 修改

    下面为主从节点实例configmap修改，有****不带删除线为新增行
    apiVersion: v1
    data:
    redis-docker-entrypoint.sh: |
    #!/bin/bash
    if [ ! -f "/redis-conf/redis.conf" ]; then
    cp /etc/redis/redis.conf /redis-conf/redis.conf
    fi
    redis-server /redis-conf/redis.conf $@
    redis.conf: |
    bind 0.0.0.0 ::
    port 6379
    daemonize no
    protected-mode no
    timeout 300
    tcp-keepalive 300
    replica-read-only yes
    replica-serve-stale-data yes
    maxclients 20000
    maxmemory 0
    maxmemory-policy noeviction
    masterauth somepassword # 删除此行配置
    requirepass somepassword # 删除此行配置
    rename-command FLUSHALL ""
    dir "/data/"
    pidfile "/data/redis.pid"
    logfile "/data/redis.log"
    kind: ConfigMap
    metadata:
    labels:
    app: redis-base-1
    type: redis
    name: redis-base-1-master
    namespace: paas-middleware

2.3 修改deployment配置参考

2.3.1 master&slaves节点的deployment的yaml修改

复制代码 
所有主从节点配置文件都修改，  有**xxx**为新增行
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
deployment.kubernetes.io/revision: "1"
  labels:
    app: redis-base-1
    type: redis
  name: redis-base-1-master
  namespace: paas-middleware
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: redis-base-1
      name: redis-base-1-master
      servicename: redis-base-1
      type: redis
      withexporter: "yes"
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "9121"
        prometheus.io/scrape: "true"
      labels:
        app: redis-base-1
        name: redis-base-1-master
        servicename: redis-base-1
        type: redis
        withexporter: "yes"
    spec:
      containers:
      - args:
        - --replica-announce-ip
        - 169.169.164.253
        - --replica-announce-port
        - "6379"
        command:
        - /etc/redis/redis-docker-entrypoint.sh
        image: harbor.somedomain/paas_middleware/redis-sentinel-main-5.0.8:latest
        imagePullPolicy: Always
        name: redis
        **env:
         - name: REDIS_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password
         - name: REDIS_MASTER_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password**

        ports:
        - containerPort: 6379
          name: client
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/redis/
          name: config
        - mountPath: /data
          name: data
        - mountPath: /redis-conf
          name: actual-config
      - args:
        - --redis.addr
        - redis://localhost:6379
        - --redis.password
        - somepassword  #密码替换成 $(REDIS_PASSWORD) 变量
        - **$(REDIS_PASSWORD)**
        - --web.listen-address
        - 0.0.0.0:9121
        image: harbor.somedomain/paas_middleware/redis-sentinel-exporter-5.0.8:latest
        imagePullPolicy: Always
        name: redis-exporter
		**env:
         - name: REDIS_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password**
        ports:
        - containerPort: 9121
          name: redis-exporter
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/hostname: 10.179.75.111
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 509
          name: redis-base-1-master
        name: config
      - hostPath:
          path: /data/redis/redis-base-1-master/data
          type: ""
        name: data
      - hostPath:
          path: /data/redis/redis-base-1-master/redis-conf
          type: ""
        name: actual-config

2.3.2 哨兵节点的deployment的yaml修改

复制代码 
所有哨兵节点配置文件都修改， 有****为新增行
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  labels:
    app: redis-base-1
    type: redis
  name: redis-base-1-sentinel-1
  namespace: paas-middleware
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: redis-base-1
      name: redis-base-1-sentinel-1
      role: sentinel
      type: redis
      withexporter: "no"
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: redis-base-1
        name: redis-base-1-sentinel-1
        role: sentinel
        type: redis
        withexporter: "no"
    spec:
      containers:
      - args:
        - --sentinel
        - announce-ip
        - 169.169.196.242
        - --replica-announce-port
        - "26379"
        command:
        - /etc/redis/redis-docker-entrypoint.sh
        image: harbor.somedomain/paas_middleware/redis-sentinel-main-5.0.8:latest
        imagePullPolicy: Always
        name: redis
        **env:
         - name: REDIS_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password
         - name: REDIS_MASTER_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password**

        ports:
        - containerPort: 26379
          name: client
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/redis/
          name: config
        - mountPath: /data
          name: data
        - mountPath: /redis-conf
          name: actual-config
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/hostname: 10.179.75.111
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 509
          name: redis-base-1-sentinel-1
        name: config
      - hostPath:
          path: /data/redis/redis-base-1-sentinel-1/data
          type: ""
        name: data
      - hostPath:
          path: /data/redis/redis-base-1-sentinel-1/redis-conf
          type: ""
        name: actual-config
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2023-11-09T03:25:41Z"
    lastUpdateTime: "2023-11-09T03:25:43Z"
    message: ReplicaSet "redis-base-1-sentinel-1-668c76f9bc" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2024-07-21T16:48:34Z"
    lastUpdateTime: "2024-07-21T16:48:34Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 3
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

2.3.3 使用命令行参数指定密码（本小节与上面两小节互斥）

  1. 哨兵节点deployment 修改
    所有哨兵节点配置文件都修改， 有****不带删除线的为新增行

    apiVersion: apps/v1
    kind: Deployment
    metadata:
    annotations:
    deployment.kubernetes.io/revision: "1"
    labels:
    app: redis-base-1
    type: redis
    name: redis-base-1-sentinel-1
    namespace: paas-middleware
    spec:
    progressDeadlineSeconds: 600
    replicas: 1
    revisionHistoryLimit: 10
    selector:
    matchLabels:
    app: redis-base-1
    name: redis-base-1-sentinel-1
    role: sentinel
    type: redis
    withexporter: "no"
    strategy:
    rollingUpdate:
    maxSurge: 25%
    maxUnavailable: 25%
    type: RollingUpdate
    template:
    metadata:
    creationTimestamp: null
    labels:
    app: redis-base-1
    name: redis-base-1-sentinel-1
    role: sentinel
    type: redis
    withexporter: "no"
    spec:
    containers:
    - args:
    - --sentinel
    - announce-ip
    - 169.169.196.242
    - --replica-announce-port
    - "26379"
    - --sentinel
    - auth-pass
    - mymaster
    - $(REDIS_PASSWORD)
    command:
    - /etc/redis/redis-docker-entrypoint.sh
    image: harbor.somedomain/paas_middleware/redis-sentinel-main-5.0.8:latest
    imagePullPolicy: Always
    name: redis
    env:
    - name: REDIS_PASSWORD
    valueFrom:
    secretKeyRef:
    name: redis-auth-secret
    key: password
    - name: REDIS_MASTER_PASSWORD
    valueFrom:
    secretKeyRef:
    name: redis-auth-secret
    key: password

    复制代码 
         ports:
     - containerPort: 26379
       name: client
       protocol: TCP
     resources: {}
     terminationMessagePath: /dev/termination-log
     terminationMessagePolicy: File
     volumeMounts:
     - mountPath: /etc/redis/
       name: config
     - mountPath: /data
       name: data
     - mountPath: /redis-conf
       name: actual-config
   dnsPolicy: ClusterFirst
   nodeSelector:
     kubernetes.io/hostname: 10.179.75.111
   restartPolicy: Always
   schedulerName: default-scheduler
   securityContext: {}
   terminationGracePeriodSeconds: 30
   volumes:
   - configMap:
       defaultMode: 509
       name: redis-base-1-sentinel-1
     name: config
   - hostPath:
       path: /data/redis/redis-base-1-sentinel-1/data
       type: ""
     name: data
   - hostPath:
       path: /data/redis/redis-base-1-sentinel-1/redis-conf
       type: ""
     name: actual-config

    status:
    availableReplicas: 1
    conditions:

    • lastTransitionTime: "2023-11-09T03:25:41Z"
      lastUpdateTime: "2023-11-09T03:25:43Z"
      message: ReplicaSet "redis-base-1-sentinel-1-668c76f9bc" has successfully progressed.
      reason: NewReplicaSetAvailable
      status: "True"
      type: Progressing
    • lastTransitionTime: "2024-07-21T16:48:34Z"
      lastUpdateTime: "2024-07-21T16:48:34Z"
      message: Deployment has minimum availability.
      reason: MinimumReplicasAvailable
      status: "True"
      type: Available
      observedGeneration: 3
      readyReplicas: 1
      replicas: 1
      updatedReplicas: 1

2、以下为主从实例deployment 配置修改

所有主从节点配置文件都修改， 有****不带删除线的为新增行

复制代码 
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
deployment.kubernetes.io/revision: "1"
  labels:
    app: redis-base-1
    type: redis
  name: redis-base-1-master
  namespace: paas-middleware
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: redis-base-1
      name: redis-base-1-master
      servicename: redis-base-1
      type: redis
      withexporter: "yes"
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "9121"
        prometheus.io/scrape: "true"
      labels:
        app: redis-base-1
        name: redis-base-1-master
        servicename: redis-base-1
        type: redis
        withexporter: "yes"
    spec:
      containers:
      - args:
        - --replica-announce-ip
        - 169.169.164.253
        - --replica-announce-port
        - "6379"
        - --**requirepass
        - $(REDIS_PASSWORD)
        - -- masterauth
        - $(REDIS_MASTER_PASSWORD)**
        command:
        - /etc/redis/redis-docker-entrypoint.sh
        image: harbor.somedomain/paas_middleware/redis-sentinel-main-5.0.8:latest
        imagePullPolicy: Always
        name: redis
        **env:
         - name: REDIS_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password
         - name: REDIS_MASTER_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password**

        ports:
        - containerPort: 6379
          name: client
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/redis/
          name: config
        - mountPath: /data
          name: data
        - mountPath: /redis-conf
          name: actual-config
      - args:
        - --redis.addr
        - redis://localhost:6379
        - --redis.password
        - somepassword  #密码替换成 $(REDIS_PASSWORD) 变量
        - **$(REDIS_PASSWORD)**
        - --web.listen-address
        - 0.0.0.0:9121
        image: harbor.somedomain/paas_middleware/redis-sentinel-exporter-5.0.8:latest
        imagePullPolicy: Always
        name: redis-exporter
		**env:
         - name: REDIS_PASSWORD
           valueFrom:
             secretKeyRef:
               name: redis-auth-secret
               key: password**
        ports:
        - containerPort: 9121
          name: redis-exporter
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/hostname: 10.179.75.111
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 509
          name: redis-base-1-master
        name: config
      - hostPath:
          path: /data/redis/redis-base-1-master/data
          type: ""
        name: data
      - hostPath:
          path: /data/redis/redis-base-1-master/redis-conf
          type: ""
        name: actual-config

2.3.4 镜像环境变量参考

https://hub.docker.com/r/bitnami/redis#configuration

https://github.com/oliver006/redis_exporter#flags

三、测试是否生效

Master节点

Slave节点

哨兵节点

测试redis-sentinel-exporter 指标抓取

四、 注意事项

  1. 所有节点configmap和deployment yaml 配置文件都按照上面修改别遗漏
  2. 修改完先在测试环境验证没有问题，再连接到连接哨兵集群进行读写测试
相关推荐
懒羊羊大王呀40 分钟前
Ubuntu20.04中 Redis 的安装和配置
linux·redis
John Song2 小时前
Redis 集群批量删除key报错 CROSSSLOT Keys in request don‘t hash to the same slot
数据库·redis·哈希算法
有个傻瓜6 小时前
PHP语言核心技术全景解析
开发语言·kubernetes·php
幻灭行度7 小时前
CKA考试知识点分享(2)---ingress
kubernetes
Zfox_11 小时前
Redis：Hash数据类型
服务器·数据库·redis·缓存·微服务·哈希算法
呼拉拉呼拉11 小时前
Redis内存淘汰策略
redis·缓存
ayuday15 小时前
Bootstrap Table强大的web数据表格渲染框架
bootstrap·bootstrap table
咖啡啡不加糖15 小时前
Redis大key产生、排查与优化实践
java·数据库·redis·后端·缓存
MickeyCV16 小时前
使用Docker部署MySQL&Redis容器与常见命令
redis·mysql·docker·容器·wsl·镜像
肥仔哥哥193016 小时前
springCloud2025+springBoot3.5.0+Nacos集成redis从nacos拉配置起服务
redis·缓存·最新boot3集成
热门推荐
01【图像处理与机器视觉】XJTU期末考点02从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑03KGG转MP3工具|非KGM文件|解密音频04海康Visionmaster-常见问题排查方法-启动阶段05YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】06【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！07Coze扣子平台完整体验和实践（附国内和国际版对比）08DeepSeek各版本说明与优缺点分析09VMware虚拟机安装Win7专业版保姆级教程（附镜像包）10R-tree详解