【ETCD】ETCD集群部署(TLS)

研究司马懿2025-11-07 12:35

ETCD集群部署(TLS)

文章目录

资源列表

操作系统 配置 主机名 IP
openEuler 2C4G etcd1 192.168.93.101
openEuler 2C4G etcd2 192.168.93.102
openEuler 2C4G etcd3 192.168.93.103

基础环境

  • 关闭防火墙
bash 复制代码 
systemctl stop firewalld
systemctl disable firewalld
  • 关闭内核安全机制
bash 复制代码 
setenforce 0
sed -i "s/^SELINUX=.*/SELINUX=disabled/g" /etc/selinux/config
  • 修改主机名
bash 复制代码 
hostnamectl set-hostname etcd1
hostnamectl set-hostname etcd2
hostnamectl set-hostname etcd3

一、部署ETCD集群

  • 在生产环境或者对高可用要求的环境下，需要使用etcd的高可用部署方式进行部署，etcd的raft协议保障各个节点数据的一致性。至少使用三台以上奇数节点，才能达到最好的集群容错。

1.1、注意事项

  • --listen-client-urls：用于监听客户端消息，必须设置为真实的IP地址，如果机器为云主机，可以设置为云主机的私有IP地址或者0.0.0.0(代表监听所有地址)，不能设置为公网IP地址。
  • --listen-peer-urls：用于监听其他member发送过来的消息，跟listen-client-urls一样，必须设置为真实IP地址，如果机器为云主机，不能设置为公网IP。
  • --initial-advertise-peer-urls：用于监听其他member同步信息，该地址其他member必须能直接访问，所以如果是云主机该地址必须设置为云主机的公网IP地址。
  • --initial-cluster集群列表：该列表中的值必须跟各个member的initial-advertise-peer-urls值一样。

1.2、添加system服务管理

  • 以下配置均在三个节点上操作
bash 复制代码 
# 创建配置文件目录
mkdir -p /etc/etcd


tar -zxvf etcd-v3.4.23-linux-amd64.tar.gz
mv etcd-v3.4.23-linux-amd64 /usr/local/etcd


tail /etc/profile -n 1
export PATH="$PATH:/usr/local/etcd"


source /etc/profile
  • 创建system服务配置
bash 复制代码 
[root@etcd ~]# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/etc/etcd/etcd.conf
ExecStart=/usr/local/etcd/etcd --config-file=/etc/etcd/etcd.conf
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

1.3、添加集群配置文件

  • 每个节点etcd配置文件主要的差异就是当前节点的IP地址和命名。部署启动方式与单接待你部署启动方式完全一致，只需要更改配置文件即可。
1.3.1、etcd1
bash 复制代码 
[root@etcd1 ~]# cat /etc/etcd/etcd.conf
# 节点名称
name: "etcd1"
# 数据存储目录
data-dir: "/data/etcd"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "http://192.168.93.101:2379"
# 监听客户端请求的地址列表
listen-client-urls: "http://192.168.93.101:2379,http://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "http://192.168.93.101:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "http://192.168.93.101:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=http://192.168.93.101:2380,etcd2=http://192.168.93.102:2380,etcd3=http://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
1.3.2、etcd2
bash 复制代码 
[root@etcd2 ~]# cat /etc/etcd/etcd.conf
# 节点名称
name: "etcd2"
# 数据存储目录
data-dir: "/data/etcd"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "http://192.168.93.102:2379"
# 监听客户端请求的地址列表
listen-client-urls: "http://192.168.93.102:2379,http://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "http://192.168.93.102:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "http://192.168.93.102:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=http://192.168.93.101:2380,etcd2=http://192.168.93.102:2380,etcd3=http://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
1.3.3、etcd3
bash 复制代码 
[root@etcd3 ~]# cat /etc/etcd/etcd.conf 
# 节点名称
name: "etcd3"
# 数据存储目录
data-dir: "/data/etcd"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "http://192.168.93.103:2379"
# 监听客户端请求的地址列表
listen-client-urls: "http://192.168.93.103:2379,http://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "http://192.168.93.103:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "http://192.168.93.103:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=http://192.168.93.101:2380,etcd2=http://192.168.93.102:2380,etcd3=http://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'

1.4、启动服务

  • 以下配置均在三个节点上操作
bash 复制代码 
systemctl daemon-reload
systemctl start etcd

1.5、验证访问

bash 复制代码 
[root@etcd1 ~]# etcdctl endpoint status --cluster -w table
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|          ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| http://192.168.93.103:2379 | 3d33f92152eaee46 |  3.4.23 |   16 kB |     false |      false |        76 |          9 |                  9 |        |
| http://192.168.93.102:2379 | 5abc4e842d3ab1d6 |  3.4.23 |   20 kB |     false |      false |        76 |          9 |                  9 |        |
| http://192.168.93.101:2379 | f416c4d7e7853c3a |  3.4.23 |   20 kB |      true |      false |        76 |          9 |                  9 |        |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

二、部署TLS加密集群

  • etcd支持通过TLS协议的加密通讯，在实际企业生产中，出于俺去那规范要求，建议开启TLS加密。TLS通道可以用于加密内部的集群通讯，也可以用于加密客户端请求。

  • etcd的TLS有两队，一对是etcd和client端的TLS配置。一对是etcd之间的peer的TLS配置。有很多方式可以创建CA证书和私钥，其中比基奥流行的两种是如下：

    • openssl
    • cfssl

  • 官方文档推荐使用cfssl生成证书

2.1、下载安装cfssl

  • 以下配置均在三个节点上操作
bash 复制代码 
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssl_1.6.3_linux_amd64
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssljson_1.6.3_linux_amd64

mv cfssl_1.6.3_linux_amd64 /usr/bin/cfssl
mv cfssljson_1.6.3_linux_amd64 /usr/bin/cfssljson
chmod +x /usr/bin/{cfssl,cfssljson}

2.2、创建默认配置文件

  • 以下配置均在三个节点上操作
bash 复制代码 
cfssl print-defaults config > ca-config.json
cfssl print-defaults csr > ca-csr.json

2.3、创建CA证书

  • 只需要在集群中其中一台节点执行即可
bash 复制代码 
[root@etcd1 ~]# cd /etc/etcd/
[root@etcd1 etcd]# mkdir -p etcdca
[root@etcd1 etcd]# cd etcdca/
  • 修改ca-config配置
bash 复制代码 
[root@etcd1 etcdca]# cat ca-config.json 
{
    "signing": {
        "default": {
            "expiry": "43800h"
        },
        "profiles": {
            "server": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth"
                ]
            },
            "client": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "peer": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}
  • 配置证书请求
bash 复制代码 
[root@etcd1 etcdca]# cat ca-csr.json 
{
  "CN": "Etcd",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "Etcd",
      "OU": "CA"
    }
  ]
}
  • 生成CA证书
bash 复制代码 
[root@etcd1 etcdca]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
[root@etcd1 etcdca]# ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem

2.4、生成服务器端证书

  • 注意hosts字段需要加上etcd全部节点的IP/主机名信息以及127.0.0.1
bash 复制代码 
[root@etcd1 etcdca]# cat server-csr.json 
{
    "CN": "server",
    "hosts": [
        "127.0.0.1",
        "192.168.93.101",
        "192.168.93.102",
        "192.168.93.103"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}


[root@etcd1 etcdca]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server-csr.json | cfssljson -bare server


[root@etcd1 etcdca]# ls server*
server.csr  server-csr.json  server-key.pem  server.pem

2.5、生成客户端证书

bash 复制代码 
[root@etcd1 etcdca]# cat client-csr.json
{
    "CN": "client",
    "hosts": [
        ""
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}


[root@etcd1 etcdca]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client-csr.json | cfssljson -bare client

[root@etcd1 etcdca]# ls client*
client.csr  client-csr.json  client-key.pem  client.pem

2.6、生成对等证书

bash 复制代码 
[root@etcd1 etcdca]# cat peer-csr.json 
{
    "CN": "peer",
    "hosts": [
        "192.168.93.101",
        "192.168.93.102",
        "192.168.93.103"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}


[root@etcd1 etcdca]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer peer-csr.json | cfssljson -bare peer


[root@etcd1 etcdca]# ls peer*
peer.csr  peer-csr.json  peer-key.pem  peer.pem

2.7、拷贝密钥到所有节点

bash 复制代码 
# 所有节点需要创建存放证书目录
cd /etc/etcd/
mkdir etcdca
bash 复制代码 
scp ca.pem server.pem server-key.pem peer.pem peer-key.pem root@192.168.93.102:/etc/etcd/etcdca/
scp ca.pem server.pem server-key.pem peer.pem peer-key.pem root@192.168.93.103:/etc/etcd/etcdca/

# 所有节点都要执行
update-ca-trust

2.8、修改etcd配置并重启etcd

  • 主要是将原本的http链接全部改为https，并指定证书密钥地址
bash 复制代码 
# etcd1
[root@etcd1 data]# cat /etc/etcd/etcd.conf 
# 节点名称
name: "etcd1"
# 数据存储目录
data-dir: "/data"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "https://192.168.93.101:2379"
# 监听客户端请求的地址列表
listen-client-urls: "https://192.168.93.101:2379,https://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "https://192.168.93.101:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "https://192.168.93.101:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=https://192.168.93.101:2380,etcd2=https://192.168.93.102:2380,etcd3=https://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
# 日志配置
logger: zap


# 客户端加密
client-transport-security:
  cert-file: "/etc/etcd/etcdca/server.pem"
  key-file: "/etc/etcd/etcdca/server-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"

# 节点加密
peer-transport-security:
  cert-file: "/etc/etcd/etcdca/peer.pem"
  key-file: "/etc/etcd/etcdca/peer-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"

# 节点加密
peer-transport-security:
  cert-file: "/etc/etcd/etcdca/peer.pem"
  key-file: "/etc/etcd/etcdca/peer-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"


# etcd2
[root@etcd2 ~]# cat /etc/etcd/etcd.conf
# 节点名称
name: "etcd2"
# 数据存储目录
data-dir: "/etc/etcd/data"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "https://192.168.93.102:2379"
# 监听客户端请求的地址列表
listen-client-urls: "https://192.168.93.102:2379,https://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "https://192.168.93.102:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "https://192.168.93.102:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=https://192.168.93.101:2380,etcd2=https://192.168.93.102:2380,etcd3=https://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
# 日志配置
logger: zap


# 客户端加密
client-transport-security:
  cert-file: "/etc/etcd/etcdca/server.pem"
  key-file: "/etc/etcd/etcdca/server-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"

# 节点加密
peer-transport-security:
  cert-file: "/etc/etcd/etcdca/peer.pem"
  key-file: "/etc/etcd/etcdca/peer-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"


# etcd3
[root@etcd3 ~]# cat /etc/etcd/etcd.conf
# 节点名称
name: "etcd3"
# 数据存储目录
data-dir: "/etc/etcd/data"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "https://192.168.93.103:2379"
# 监听客户端请求的地址列表
listen-client-urls: "https://192.168.93.103:2379,https://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "https://192.168.93.103:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "https://192.168.93.103:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=https://192.168.93.101:2380,etcd2=https://192.168.93.102:2380,etcd3=https://192.168.93.103:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
# 日志配置
logger: zap


# 客户端加密
client-transport-security:
  cert-file: "/etc/etcd/etcdca/server.pem"
  key-file: "/etc/etcd/etcdca/server-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"

# 节点加密
peer-transport-security:
  cert-file: "/etc/etcd/etcdca/peer.pem"
  key-file: "/etc/etcd/etcdca/peer-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/etcdca/ca.pem"
  • 重新之前先把老的数据删除，不然可能会出现无法连接等问题
bash 复制代码 
# 所有节点都需要操作
cd /etc/etcd/data/
rm -rf member/
systemctl restart etcd.service

2.9、验证

bash 复制代码 
[root@etcd3 etcdca]# etcdctl --endpoints=https://192.168.93.103:2379 --cacert=ca.pem --cert=client.pem --key=client-key.pem endpoint status --cluster -w table
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|          ENDPOINT           |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.93.101:2379 |  669dcba8ba3cf92 |  3.4.23 |   16 kB |     false |      false |       276 |         18 |                 18 |        |
| https://192.168.93.103:2379 | 959726a465b25a58 |  3.4.23 |   20 kB |      true |      false |       276 |         18 |                 18 |        |
| https://192.168.93.102:2379 | 99f09f4bbc98d0ce |  3.4.23 |   20 kB |     false |      false |       276 |         18 |                 18 |        |
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
  • 写入数据测试
bash 复制代码 
[root@etcd3 etcdca]# etcdctl --endpoints=https://192.168.93.103:2379 --cacert=ca.pem --cert=client.pem --key=client-key.pem put newtest "hello world"
OK
  • 读取数据测试
bash 复制代码 
[root@etcd3 etcdca]# etcdctl --endpoints=https://192.168.93.101:2379 --cacert=ca.pem --cert=client.pem --key=client-key.pem get newtest
newtest
hello world
相关推荐
行者游学2 小时前
ETCD 备份脚本
数据库·etcd
hu1j2 小时前
[HTB] 靶机学习（十二）Eureka
学习·安全·web安全·网络安全·云原生·eureka
没有bug.的程序员2 小时前
Eureka 注册中心原理与服务注册发现机制
java·spring·云原生·eureka·架构·注册中心·服务注册发现
研究司马懿2 小时前
【ETCD】ETCD单节点二进制部署(TLS)
数据库·chrome·云原生·运维开发·etcd·argocd·gitops
行者游学2 小时前
ETCD 压力测试脚本
数据库·压力测试·etcd
春生野草3 小时前
SpringBoot配置文件
java·数据库·spring boot
奇点 ♡3 小时前
MySQL基础题
数据库·sql·mysql
唐古乌梁海3 小时前
【mysql】MySQL 数据库迁移
数据库·mysql·adb
啊吧怪不啊吧3 小时前
SQL之表的时间类内置函数详解
大数据·服务器·数据库·sql
热门推荐
01GitHub 镜像站点02综合整理：pdf预览显示：你尝试预览的文件可能对你的计算机有害。如果你信任此文件以及其来源，请打开此文件以看其内容，如何解决以正常预览文件03UV安装并设置国内源04Linux下V2Ray安装配置指南05npm使用国内淘宝镜像的方法06安娜的档案(Anna’s Archive) 镜像网站/国内最新可访问入口（持续更新）07BongoCat - 跨平台键盘猫动画工具08NVIDIA显卡驱动、CUDA、cuDNN 和 TensorRT 版本匹配指南09《大数据技术原理与应用》实验报告三 熟悉HBase常用操作10GitLab 零基础入门指南：从安装到项目管理全流程