K8S集群部署问题及处理记录

问题一：重启kubelet失败或状态不对

kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: activating (auto-restart) (Result: exit-code) since 四 2023-11-16 12:10:53 CST; 1s ago
     Docs: https://kubernetes.io/docs/
  Process: 13824 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
 Main PID: 13824 (code=exited, status=1/FAILURE)

使用journalctl -u kubelet --no-pager 查看启动报错日志

​​journalctl​​​ 的 ​​-u​​​ 参数可以指定服务进行过滤，这样可以屏蔽掉其他无关日志。 ​​--no-pager​​ 参数可以一次性输出日志，当然如果你只是在线查看，则可以不用这个参数

解决方式一：关闭SWAP尝试

原因是 ​​kubelet​​​ 默认不支持 ​​swap​​​ ，所以需要关闭swap，或者在 kubelet 启动时传递 ​​--fail-swap-on​​ 参数；

关闭swap

swapoff -a # 临时

sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久

解决方式二：驱动问题

查看docker驱动

docker info | grep Cgroup

查看kubelet驱动

cat /var/lib/kubelet/config.yaml | grep cgroup

修改docker驱动，查看/etc/docker/daemon.json文件

{
  "registry-mirrors": ["http://hub-mirror.c.163.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}

重启containerd以及kubelet

解决方式三：10-kubeadm.conf文件

查看你的/usr/lib/systemd/system/kubelet.service.d/下是否有这个文件

10-kubeadm.conf`是一个配置文件，它位于`/etc/systemd/system/kubelet.service.d/`目录下。这个文件的主要作用是指定Kubelet的动态配置。

在Kubeadm初始化集群后，Kubelet将以动态方式获取其配置。为了实现这一功能，应设置标志`--dynamic-config-dir=/var/lib/kubelet/config/dynamic`于`10-kubeadm.conf`文件中。

此外，Kubelet的其他关键配置，如唯一标识符的KubeConfig文件（位于`/etc/kubernetes/kubelet.conf`），包含组件配置的文件（位于`/var/lib/kubelet/config.yaml`），以及包含动态环境的文件（来自`/var/lib/kubelet/kubeadm-flags.env`）等，都对Kubelet的运行起到重要作用。

没有的话就创建

# Note: This dropin only works with kubeadm and kubelet v2.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

然后重新启动

systemctl daemon-reload
systemctl restart kubelet.service

解决方式四：cri

cat /etc/containerd/config.toml

#   Copyright 2018-2022 Docker Inc.

#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at

#       http://www.apache.org/licenses/LICENSE-2.0

#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

#disabled_plugins = ["cri"]

#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0

#[grpc]
#  address = "/run/containerd/containerd.sock"
#  uid = 0
#  gid = 0

#[debug]
#  address = "/run/containerd/debug.sock"
#  uid = 0
#  gid = 0
#  level = "info"

CRI，全称Container Runtime Interface，中文为容器运行时接口，是Kubernetes v1.5版本引入的一个关键组件。它的主要作用是将Kubelet与容器运行时进行解耦。具体来说，CRI将原来完全面向Pod级别的内部接口拆分成面向Sandbox和Container的gRPC接口，并将镜像管理和容器管理分离到不同的服务。

CRI是一个插件接口，这使得kubelet能够使用各种容器运行时而无需重新编译集群组件。换句话说，只要在集群中的每个节点上都有一个可以正常工作的容器运行时，kubelet就能够启动Pod及其容器。这大大提高了系统的灵活性和可扩展性。

重启 containerd和kubelet