证书更新后 K8s Master状态NotReady

证书更新后 K8s Master状态NotReady

shell 复制代码
[root@k8s-master01:7 /var/lib/kubelet/pki]# kubectl get nodes
NAME                             STATUS     ROLES    AGE     VERSION
k8s-master01.amngrvmm.dc01.scf   Ready      master   5y65d   v1.17.6
k8s-master02.amngrvmm.dc01.scf   NotReady   master   5y65d   v1.17.6
k8s-master03.amngrvmm.dc01.scf   Ready      master   5y65d   v1.17.6
k8s-node01.amngrvmm.dc01.scf     Ready      <none>   5y65d   v1.17.6
k8s-node02.amngrvmm.dc01.scf     Ready      <none>   608d    v1.17.6
k8s-node03.amngrvmm.dc01.scf     Ready      <none>   608d    v1.17.6

查看日志

shell 复制代码
[root@k8s-master02:4 ~]# journalctl -u kubelet -f
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Service RestartSec=10s expired, scheduling restart.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 52.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: Started kubelet: The Kubernetes Node Agent.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https:s.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https:s.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.724653    2797 server.go:416] Version: v1.17.6
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.725245    2797 plugins.go:100] No cloud provider specified.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.725308    2797 server.go:821] Client rotation is on, will bootstrap in background
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: E0805 15:45:49.730326    2797 bootstrap.go:265] part of the existing bootstrap client certificate is expired: 2024-12-05 07:11:30 +0000 UTC
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: F0805 15:45:49.730409    2797 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubeletuch file or directory
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Failed with result 'exit-code'.

kubelet 客户端证书过期

进入 /var/lib/kubelet/pki文件夹查看,kubelet的证书还是老证书 kubelet-client-2024-08-19-08-12-55.pem

shell 复制代码
[root@k8s-master02:15 /var/lib/kubelet/pki]# ll
total 32
-rw------- 1 root root 1098 Jun  2  2020 kubelet-client-2020-06-02-11-23-02.pem
-rw------- 1 root root 1098 Apr  3  2021 kubelet-client-2021-04-03-00-58-46.pem
-rw------- 1 root root 1098 Dec 26  2021 kubelet-client-2021-12-26-17-48-17.pem
-rw------- 1 root root 1098 Oct 23  2022 kubelet-client-2022-10-23-20-49-54.pem
-rw------- 1 root root 1098 Dec  6  2023 kubelet-client-2023-12-06-15-15-55.pem
-rw------- 1 root root 1098 Aug 19  2024 kubelet-client-2024-08-19-08-12-55.pem
lrwxrwxrwx 1 root root   59 Aug 19  2024 kubelet-client-current.pem -> /var/lib/kubelet/pki/kubelet-client-2024-08-19-08-12-55.pem
-rw-r--r-- 1 root root 2315 Jun  2  2020 kubelet.crt
-rw------- 1 root root 1679 Jun  2  2020 kubelet.key

进入**/etc/kubernetes**文件夹

备份一下kubelet.conf

将kubelet.conf中的client-certificate 、client-key-data替换为admin.conf中的client-certificate-data、client-key-data

shell 复制代码
[root@k8s-master02:20 /etc/kubernetes]# ls
admin.conf  controller-manager.conf  kubelet.conf  manifests  pki  scheduler.conf
[root@k8s-master02:21 /etc/kubernetes]# cp kubelet.conf kubelet.conf.back
[root@k8s-master02:22 /etc/kubernetes]# vim kubelet.conf

替换完成后,可以看到 /var/lib/kubelet/pki 文件夹下生成了新的kubelet-client证书kubelet-client-2025-08-05-15-57-18.pem

shell 复制代码
[root@k8s-master02:25 /var/lib/kubelet/pki]# ll
total 36
-rw------- 1 root root 1098 Jun  2  2020 kubelet-client-2020-06-02-11-23-02.pem
-rw------- 1 root root 1098 Apr  3  2021 kubelet-client-2021-04-03-00-58-46.pem
-rw------- 1 root root 1098 Dec 26  2021 kubelet-client-2021-12-26-17-48-17.pem
-rw------- 1 root root 1098 Oct 23  2022 kubelet-client-2022-10-23-20-49-54.pem
-rw------- 1 root root 1098 Dec  6  2023 kubelet-client-2023-12-06-15-15-55.pem
-rw------- 1 root root 1098 Aug 19  2024 kubelet-client-2024-08-19-08-12-55.pem
-rw------- 1 root root 1098 Aug  5 15:57 kubelet-client-2025-08-05-15-57-18.pem
lrwxrwxrwx 1 root root   59 Aug  5 15:57 kubelet-client-current.pem -> /var/lib/kubelet/pki/kubelet-client-2025-08-05-15-57-18.pem
-rw-r--r-- 1 root root 2315 Jun  2  2020 kubelet.crt
-rw------- 1 root root 1679 Jun  2  2020 kubelet.key

恢复kubelet.conf 中的原始配置,不恢复也可以正常使用

重启kubelet或者重启机器

再次查看master02状态 Ready,恢复正常

shell 复制代码
[root@k8s-master01:8 /var/lib/kubelet/pki]# kubectl get nodes
NAME                             STATUS   ROLES    AGE     VERSION
k8s-master01.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-master02.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-master03.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-node01.amngrvmm.dc01.scf     Ready    <none>   5y65d   v1.17.6
k8s-node02.amngrvmm.dc01.scf     Ready    <none>   608d    v1.17.6
k8s-node03.amngrvmm.dc01.scf     Ready    <none>   608d    v1.17.6
相关推荐
运维开发王义杰2 小时前
Kubernetes: 解构Karpenter NodePool, 云原生时代的弹性节点管理艺术
云原生·容器·kubernetes
上邪o_O2 小时前
从零开始部署 Kubernetes Dashboard:可视化管理你的集群
云原生·kubernetes
草莓田田圈~5 小时前
kubernetes-ubuntu24.04操作系统部署k8s集群
云原生·容器·kubernetes
Brilliantee4045 小时前
K8s 二次开发漫游录
云原生·容器·kubernetes·operator·k8s二次开发
007php0076 小时前
Go 语言常用命令使用与总结
java·linux·服务器·前端·数据库·docker·容器
你是我的日月星河8 小时前
Docker部署单节点使用KRaft存储数据的Kafka与可视化界面Kafka-Map
docker·容器·kafka
Cyber4K8 小时前
企业级-搭建CICD(持续集成持续交付)实验手册
ci/cd·云原生·容器
搞不懂语言的程序员9 小时前
docker 搭建zookper集群,快照虚拟机多机模拟
运维·docker·容器
Rysxt_9 小时前
Docker Swarm 与 Kubernetes (K8s) 全面对比教程
docker·容器·kubernetes
Britz_Kevin12 小时前
从零开始的云计算生活——第五十四天,悬梁刺股,kubernetes模块之组件与网络
网络·kubernetes·生活