k8s-1.22.3集群etcd备份与恢复

归海听雪2023-08-09 20:59

一、环境准备

注：请在测试环境下验证操作

CentOS Linux release 7.7.1908 (Core) 3.10.0-1062.el7.x86_64

kubeadm-1.22.3-0.x86_64

kubelet-1.22.3-0.x86_64

kubectl-1.22.3-0.x86_64

kubernetes-cni-0.8.7-0.x86_64

|--------------|---------------|---------------|
| 主机名 | IP | VIP |
| k8s-master01 | 192.168.10.61 | 192.168.10.70 |
| k8s-master02 | 192.168.10.62 | 192.168.10.70 |
| k8s-master03 | 192.168.10.63 | 192.168.10.70 |
| k8s-node01 | 192.168.10.64 | 192.168.10.70 |
| k8s-node02 | 192.168.10.65 | 192.168.10.70 |

二、安装etcdctl工具

1、yum安装

bash 复制代码 
yum install -y etcd

2、或者直接下载静态编译的包

bash 复制代码 
wget https://github.com/etcd-io/etcd/releases/download/v3.4.14/etcd-v3.4.14-linux-amd64.tar.gz
tar -zxf etcd-v3.4.14-linux-amd64.tar.gz
cd etcd-v3.4.14-linux-amd64
cp etcdctl /usr/local/bin

注：etcd最新的API版本是v3，与v2相比，v3更高效更清晰。k8s默认使用的etcd V3版本API，ectdctl默认使用V2版本API。要想使用v3，需要设置环境变量export ETCDCTL_API=3临时更改为V3或者在 /etc/profile后在里面添加export ETCDCTL_API=3，然后执行source /etc/profile则永久更改为V3。

bash 复制代码 
echo "export ETCDCTL_API=3" >>/etc/profile
source /etc/profile

三、查看etcd节点和状态

注：根据自已的环境指定证书路径

1、查看etcd节点

bash 复制代码 
etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --write-out=table --endpoints=192.168.10.61:2379,192.168.10.63:2379,192.168.10.62:2379 member list
+------------------+---------+--------------+----------------------------+----------------------------+------------+
|        ID        | STATUS  |     NAME     |         PEER ADDRS         |        CLIENT ADDRS        | IS LEARNER |
+------------------+---------+--------------+----------------------------+----------------------------+------------+
| 3c3f0bd3bdd4ab17 | started | k8s-master01 | https://192.168.10.61:2380 | https://192.168.10.61:2379 |      false |
| 8f9d6f521fe8bcf3 | started | k8s-master03 | https://192.168.10.63:2380 | https://192.168.10.63:2379 |      false |
| c23c5081dc6638ca | started | k8s-master02 | https://192.168.10.62:2380 | https://192.168.10.62:2379 |      false |
+------------------+---------+--------------+----------------------------+----------------------------+------------+

2、查看etcd节点状态

bash 复制代码 
etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --write-out=table --endpoints=192.168.10.61:2379,192.168.10.63:2379,192.168.10.62:2379 endpoint  status
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|      ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 192.168.10.61:2379 | 3c3f0bd3bdd4ab17 |   3.5.6 |  4.4 MB |     false |      false |         4 |     892897 |             892897 |        |
| 192.168.10.63:2379 | 8f9d6f521fe8bcf3 |   3.5.6 |  4.4 MB |     false |      false |         4 |     892897 |             892897 |        |
| 192.168.10.62:2379 | c23c5081dc6638ca |   3.5.6 |  4.4 MB |      true |      false |         4 |     892897 |             892897 |        |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

四、备份etcd数据

1、新建验证数据

注：为了验证恢复数据是否正确，可以在备份之前新建一个namespace

bash 复制代码 
kubectl create ns test-ns

#查看新建的ns

bash 复制代码 
# kubectl get ns
NAME                   STATUS   AGE
default                Active   151d
ingress-nginx          Active   151d
kube-node-lease        Active   151d
kube-public            Active   151d
kube-system            Active   151d
kubernetes-dashboard   Active   151d
test-ns                Active   47s

2、备份

注：备份只需要找其中一个master节点的etcd进行备份就可以。

bash 复制代码 
etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --write-out=table --endpoints=192.168.10.61:2379 snapshot save /opt/etcd-snapshot.db

3、备份完后，再删除新建的ns

bash 复制代码 
kubectl delete ns test-ns

五、恢复etcd数据

注：k8s集群中如果有多个etcd组成的集群，那么每个etcd都需要进行独立恢复

#在恢复前需要把每个节点的master上的服务先停掉，以免有新的数据写入，并且要把默认的etcd数据目录改名

bash 复制代码 
mv /etc/kubernetes/manifests/ /etc/kubernetes/manifests.bak
mv /var/lib/etcd /var/lib/etcd.bak

#以下证书和key、name等相关信息可以在 /etc/kubernetes/manifests/etcd.yaml查到

bash 复制代码 
    - --advertise-client-urls=https://192.168.10.61:2379
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --client-cert-auth=true
    - --data-dir=/var/lib/etcd
    - --experimental-initial-corrupt-check=true
    - --initial-advertise-peer-urls=https://192.168.10.61:2380
    - --initial-cluster=k8s-master01=https://192.168.10.61:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key
    - --listen-client-urls=https://127.0.0.1:2379,https://192.168.10.61:2379
    - --listen-metrics-urls=http://127.0.0.1:2381
    - --listen-peer-urls=https://192.168.10.61:2380
    - --name=k8s-master01
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --snapshot-count=10000
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

1、恢复k8s-master01的etcd数据

#首先停掉服务

bash 复制代码 
mv /etc/kubernetes/manifests/ /etc/kubernetes/manifests.bak
mv /var/lib/etcd /var/lib/etcd.bak

#恢复数据，要把上面备份的etcd数据分别上传到另外2台

bash 复制代码 
etcdctl snapshot restore /opt/etcd-snapshot.db --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --name k8s-master01 --initial-cluster "k8s-master02=https://192.168.10.62:2380,k8s-master01=https://192.168.10.61:2380,k8s-master03=https://192.168.10.63:2380" --initial-advertise-peer-urls https://192.168.10.61:2380 --data-dir=/var/lib/etcd

2、恢复k8s-master02的etcd数据

#首先停掉服务

bash 复制代码 
mv /etc/kubernetes/manifests/ /etc/kubernetes/manifests.bak
mv /var/lib/etcd /var/lib/etcd.bak

#恢复数据

bash 复制代码 
etcdctl snapshot restore /opt/etcd-snapshot.db --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --name k8s-master02 --initial-cluster "k8s-master02=https://192.168.10.62:2380,k8s-master01=https://192.168.10.61:2380,k8s-master03=https://192.168.10.63:2380" --initial-advertise-peer-urls https://192.168.10.62:2380 --data-dir=/var/lib/etcd

3、恢复k8s-master03的etcd数据

#首先停掉服务

bash 复制代码 
mv /etc/kubernetes/manifests/ /etc/kubernetes/manifests.bak
mv /var/lib/etcd /var/lib/etcd.bak

#恢复数据

bash 复制代码 
etcdctl snapshot restore /opt/etcd-snapshot.db --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --name k8s-master03 --initial-cluster "k8s-master02=https://192.168.10.62:2380,k8s-master01=https://192.168.10.61:2380,k8s-master03=https://192.168.10.63:2380" --initial-advertise-peer-urls https://192.168.10.63:2380 --data-dir=/var/lib/etcd

4、恢复服务，3台master都需要操作

注：每个执行恢复数据操作后，都会新生成/var/lib/etcd数据目录

bash 复制代码 
mv /etc/kubernetes/manifests.bak /etc/kubernetes/manifests

5、查看数据是否恢复

bash 复制代码 
# kubectl get ns
NAME                   STATUS   AGE
default                Active   151d
ingress-nginx          Active   151d
kube-node-lease        Active   151d
kube-public            Active   151d
kube-system            Active   151d
kubernetes-dashboard   Active   151d
test-ns                Active   47s
相关推荐
aherhuo12 小时前
kubevirt网络
linux·云原生·容器·kubernetes
陌北v113 小时前
Docker Compose 配置指南
运维·docker·容器·docker-compose
catoop13 小时前
K8s 无头服务（Headless Service）
云原生·容器·kubernetes
阿里嘎多学长13 小时前
docker怎么部署高斯数据库
运维·数据库·docker·容器
liuxuzxx14 小时前
1.24.1-Istio安装
kubernetes·istio·service mesh
G_whang15 小时前
windos 安装docker
运维·docker·容器
道一云黑板报15 小时前
Flink集群批作业实践：七析BI批作业执行
大数据·分布式·数据分析·flink·kubernetes
运维小文15 小时前
K8S中的PV、PVC介绍和使用
docker·云原生·容器·kubernetes·存储
ζั͡山 ั͡有扶苏 ั͡✾16 小时前
Kubeadm+Containerd部署k8s(v1.28.2)集群（非高可用版）
云原生·容器·kubernetes
Hadoop_Liang16 小时前
Kubernetes ConfigMap的创建与使用
云原生·容器·kubernetes
热门推荐
01【一文读懂】NTN（非地面网络）技术介绍02优化手机性能，解决卡顿问题：关闭这3个微信开关，释放内存空间03玄机平台应急响应—webshell查杀04Cube-Studio：开源大模型全链路一站式中台05基于大语言模型（LLM）的合成数据生成、策展和评估的综述06(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明07组基轨迹建模 GBTM的介绍与实现（Stata 或 R）08HCIA-datacom数通题库和录播视频资料09Docker 夺命连环 15 问10ARM学习（31）编译器对overlay方式的支持