证书更新后 K8s Master状态NotReady

证书更新后 K8s Master状态NotReady

shell 复制代码
[root@k8s-master01:7 /var/lib/kubelet/pki]# kubectl get nodes
NAME                             STATUS     ROLES    AGE     VERSION
k8s-master01.amngrvmm.dc01.scf   Ready      master   5y65d   v1.17.6
k8s-master02.amngrvmm.dc01.scf   NotReady   master   5y65d   v1.17.6
k8s-master03.amngrvmm.dc01.scf   Ready      master   5y65d   v1.17.6
k8s-node01.amngrvmm.dc01.scf     Ready      <none>   5y65d   v1.17.6
k8s-node02.amngrvmm.dc01.scf     Ready      <none>   608d    v1.17.6
k8s-node03.amngrvmm.dc01.scf     Ready      <none>   608d    v1.17.6

查看日志

shell 复制代码
[root@k8s-master02:4 ~]# journalctl -u kubelet -f
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Service RestartSec=10s expired, scheduling restart.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 52.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: Started kubelet: The Kubernetes Node Agent.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https:s.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https:s.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.724653    2797 server.go:416] Version: v1.17.6
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.725245    2797 plugins.go:100] No cloud provider specified.
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: I0805 15:45:49.725308    2797 server.go:821] Client rotation is on, will bootstrap in background
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: E0805 15:45:49.730326    2797 bootstrap.go:265] part of the existing bootstrap client certificate is expired: 2024-12-05 07:11:30 +0000 UTC
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf kubelet[2797]: F0805 15:45:49.730409    2797 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubeletuch file or directory
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Aug 05 15:45:49 k8s-master02.amngrvmm.dc01.scf systemd[1]: kubelet.service: Failed with result 'exit-code'.

kubelet 客户端证书过期

进入 /var/lib/kubelet/pki文件夹查看,kubelet的证书还是老证书 kubelet-client-2024-08-19-08-12-55.pem

shell 复制代码
[root@k8s-master02:15 /var/lib/kubelet/pki]# ll
total 32
-rw------- 1 root root 1098 Jun  2  2020 kubelet-client-2020-06-02-11-23-02.pem
-rw------- 1 root root 1098 Apr  3  2021 kubelet-client-2021-04-03-00-58-46.pem
-rw------- 1 root root 1098 Dec 26  2021 kubelet-client-2021-12-26-17-48-17.pem
-rw------- 1 root root 1098 Oct 23  2022 kubelet-client-2022-10-23-20-49-54.pem
-rw------- 1 root root 1098 Dec  6  2023 kubelet-client-2023-12-06-15-15-55.pem
-rw------- 1 root root 1098 Aug 19  2024 kubelet-client-2024-08-19-08-12-55.pem
lrwxrwxrwx 1 root root   59 Aug 19  2024 kubelet-client-current.pem -> /var/lib/kubelet/pki/kubelet-client-2024-08-19-08-12-55.pem
-rw-r--r-- 1 root root 2315 Jun  2  2020 kubelet.crt
-rw------- 1 root root 1679 Jun  2  2020 kubelet.key

进入**/etc/kubernetes**文件夹

备份一下kubelet.conf

将kubelet.conf中的client-certificate 、client-key-data替换为admin.conf中的client-certificate-data、client-key-data

shell 复制代码
[root@k8s-master02:20 /etc/kubernetes]# ls
admin.conf  controller-manager.conf  kubelet.conf  manifests  pki  scheduler.conf
[root@k8s-master02:21 /etc/kubernetes]# cp kubelet.conf kubelet.conf.back
[root@k8s-master02:22 /etc/kubernetes]# vim kubelet.conf

替换完成后,可以看到 /var/lib/kubelet/pki 文件夹下生成了新的kubelet-client证书kubelet-client-2025-08-05-15-57-18.pem

shell 复制代码
[root@k8s-master02:25 /var/lib/kubelet/pki]# ll
total 36
-rw------- 1 root root 1098 Jun  2  2020 kubelet-client-2020-06-02-11-23-02.pem
-rw------- 1 root root 1098 Apr  3  2021 kubelet-client-2021-04-03-00-58-46.pem
-rw------- 1 root root 1098 Dec 26  2021 kubelet-client-2021-12-26-17-48-17.pem
-rw------- 1 root root 1098 Oct 23  2022 kubelet-client-2022-10-23-20-49-54.pem
-rw------- 1 root root 1098 Dec  6  2023 kubelet-client-2023-12-06-15-15-55.pem
-rw------- 1 root root 1098 Aug 19  2024 kubelet-client-2024-08-19-08-12-55.pem
-rw------- 1 root root 1098 Aug  5 15:57 kubelet-client-2025-08-05-15-57-18.pem
lrwxrwxrwx 1 root root   59 Aug  5 15:57 kubelet-client-current.pem -> /var/lib/kubelet/pki/kubelet-client-2025-08-05-15-57-18.pem
-rw-r--r-- 1 root root 2315 Jun  2  2020 kubelet.crt
-rw------- 1 root root 1679 Jun  2  2020 kubelet.key

恢复kubelet.conf 中的原始配置,不恢复也可以正常使用

重启kubelet或者重启机器

再次查看master02状态 Ready,恢复正常

shell 复制代码
[root@k8s-master01:8 /var/lib/kubelet/pki]# kubectl get nodes
NAME                             STATUS   ROLES    AGE     VERSION
k8s-master01.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-master02.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-master03.amngrvmm.dc01.scf   Ready    master   5y65d   v1.17.6
k8s-node01.amngrvmm.dc01.scf     Ready    <none>   5y65d   v1.17.6
k8s-node02.amngrvmm.dc01.scf     Ready    <none>   608d    v1.17.6
k8s-node03.amngrvmm.dc01.scf     Ready    <none>   608d    v1.17.6
相关推荐
千寻girling34 分钟前
记录第一次学习 Docker
学习·docker·容器
迷糊小面包2 小时前
Docker Hadopp集群版部署搭建及常规问题解疑
运维·docker·容器
烁3473 小时前
Docker
运维·docker·容器
网络中的夜鹰4 小时前
轩辕镜像一键安装Docker和Docker Compose脚本
运维·docker·容器
lihongbao804 小时前
kuboard v3创建用户分配命名空间
kubernetes·kuboard
江湖有缘7 小时前
Docker部署HamsterBase Tasks任务管理工具
运维·docker·容器
很楠爱上7 小时前
Docker 从入门到实战:核心概念、微服务编排与环境移植完全指南
docker·微服务·容器
Qres8218 小时前
docker & WSL & Ubuntu安装记录
ubuntu·docker·容器·wsl
java_cj8 小时前
从kubectl源码学pprof:生产环境性能分析的实战指南
运维·云原生·容器·kubernetes
爱吃龙利鱼10 小时前
K8s 监控实战:victoria-metrics-k8s-stack 高可用部署,资源占用直降 70%,比 Prometheus 省 5 倍磁盘
docker·kubernetes·prometheus