【云原生】最新版Kubernetes集群基于Containerd部署

文章目录

Kubernetes集群基于Containerd部署
- 资源列表
- 基础环境
- 一、基础环境准备
- 二、准备Containerd容器运行时
- 三、部署Kubernetes集群
- 四、部署网络插件（CNI）
- - 4.1、拉取必要镜像
  - 4.2、部署网络插件
  - 4.2、查看节点状态
  - 4.3、查看组件状态
  - 4.4、查看所有Pod状态
  - [4.5 解决异常pod状态](#4.5 解决异常pod状态)
- 五、Containerd基本操作
- - 5.1、Crictl镜像管理
  - 5.2、Crictl容器管理
  - [5.3、Crictl Pod管理](#5.3、Crictl Pod管理)

Kubernetes集群基于Containerd部署

资源列表

操作系统	配置	主机名	IP
CentOS 7.9.2009	2C 2G	master	192.168.47.102
CentOS 7.9.2009	2C 2G	worker	192.168.47.103

基础环境

关闭防火墙

systemctl stop firewalld
#关闭开机自启动
systemctl disable firewalld
关闭内核安全机制

setenforce 0
sed -i "s/^SELINUX=.*/SELINUX=disabled/g" /etc/selinux/config
修改主机名

192.168.47.102机器执行

hostnamectl set-hostname master

192.168.47.103机器执行

hostnamectl set-hostname worker01

一、基础环境准备

待安装的每台机器都需要进行操作，以下只对演示环境的机器进行配置并演示。

1.1、关闭Swap分区

集群的每台机器都需要关闭swap分区

# 临时关闭
swapoff -a 
# 永久关闭
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

1.2、添加hosts解析

cat >> /etc/hosts << EOF
192.168.47.102 master
192.168.47.103 worker01
EOF

1.3、桥接的IPv4流量传递给iptables的链

# 加载 overlay 内核模块
modprobe overlay
# 加载 br_netfilter 模块
modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF


 sysctl --system

二、准备Containerd容器运行时

待安装的每台机器都需要进行操作，以下只对演示环境的机器进行配置并演示。

2.1、安装Containerd

在线安装

添加 docker 源，containerd也在docker源内的

cat <<EOF | sudo tee /etc/yum.repos.d/docker-ce.repo
[docker]
name=docker-ce
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
EOF

快速建立 yum 缓存

yum makecache fast

安装containerd

列出所有containerd版本

yum list containerd.io --showduplicates
yum -y install containerd.io-1.6.6-3.1.el7.x86_64

2.2、配置Containerd

# 生成配置文件
mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml 


# 修改/etc/containerd/config.toml文件中sandbox_image的值
grep 'sandbox_image' /etc/containerd/config.toml 
# 在原有的基础上进行修改，
    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"

2.3、启动Containerd

 systemctl enable containerd
 systemctl start containerd

三、部署Kubernetes集群

3.1、安装Kubeadm工具

所有节点都要操作

在线安装

cat << EOF >> /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

快速建立yum缓存

yum makecache fast

安装 kubectl：命令行管理工具、kubeadm：安装K8S集群工具、kubelet管理容器工具

yum install -y kubelet-1.27.0 kubeadm-1.27.0 kubectl-1.27.0

切记这个时候不要启动，只需要设置为开机自启

systemctl enable kubelet.service

3.2、配置crictl工具

crictl是CRI兼容的容器运行时命令行接口。你可以使用它来检查和调试Kubernetes节点上的容器运行时和应用程序。crictl和它的源代码在cri-tools
代码库
更好 Containerd后，以上我们常用的docker命令也不再使用了，取而代之恶的分别是 crictl 和 ctr 两个命令行客户端
crictl是遵循CRI接口规范的一个命令行工具，通常用它来检查和管理kubelet节点上的容器运行时和镜像
ctr是containerd的一个客户端工具

所有节点都要操作

cat << EOF >> /etc/crictl.yaml
runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
timeout: 10
debug: false
EOF

3.3、配置Containerd镜像加速器

所有节点都要操作

# 在/etc/containerd/config.toml找到[plugins."io.containerd.grpc.v1.cri".registry.mirrors] ，默认在153行左右，在此行下面添加两行配置，阿里云的镜像加速器
vim /etc/containerd/config.toml 
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors].
  #亲测有用
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
        endpoint = ["https://docker.anyhub.us.kg","https://dockerhub.jobcher.com","https://docker.ckyl.me","https://hub.uuuadc.top","https://docker.1panel.live","https://hub.rat.dev/","https://docker.chenby.cn","https://docker.m.daocloud.io","https://docker.awsl9527.cn","https://huecker.io","https://dockerhub.timeweb.cloud","https://noohub.ru"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]
            endpoint=["https://45hrqeao.mirror.aliyuncs.com", "https://k8s.m.daocloud.io", "https://docker.mirrors.ustc.edu.cn","https://hub-mirror.c.163.com"]
     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
      endpoint = ["registry.aliyuncs.com/google_containers"]


# 保存退出，刷新服务
 systemctl restart containerd

3.4、测试crictl工具是否可用

# 以拉取一个pause镜像进行演示是否可用
crictl pull registry.k8s.io/pause:3.9
Image is up to date for sha256:e6f1816883972d4be47bd48879a08919b96afcd344132622e4d444987919323c

# 查看镜像
[root@localhost containerd]# crictl images
IMAGE                   TAG                 IMAGE ID            SIZE
registry.k8s.io/pause   3.9                 e6f1816883972       322kB

3.5、初始化Master节点

在master节点上操作

# 生成配置文件
kubeadm config print init-defaults > kubeadm-init.yaml


# 修改kubeadm-init.yaml文件的advertiseAddress、name、imageRepository，添加Pod网络podSubnet
 vim kubeadm-init.yaml
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.47.102
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: master
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.27.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: 10.244.0.0/16
scheduler: {}

# 初始化集群
[root@admin01 ~]# kubeadm init --config=kubeadm-init.yaml
[init] Using Kubernetes version: v1.27.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W0216 21:15:32.473341   59711 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.0, falling back to the nearest etcd version (3.5.7-0)
W0216 21:15:32.761039   59711 checks.go:835] detected that the sandbox image "registry.aliyuncs.com/google_containers/pause:3.9" of the container runtime is inconsistent with that used by kubeadm. It is recommended that using "registry.k8s.io/pause:3.9" as the CRI sandbox image.
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 192.168.47.102]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.47.102 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.47.102 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
W0216 21:15:38.631109   59711 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.0, falling back to the nearest etcd version (3.5.7-0)
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 10.513140 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.47.102:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:0d2e1f77f8531ecfbccdace3dd0f78507fe31c6ce1da02cecf5f35788a8ec613


# 初始化成功以后要根据提示执行以下3个命令，才可以操作集群
[root@master ~]# mkdir -p $HOME/.kube
[root@master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

待添加的节点执行如下命令即可：

kubeadm join 192.168.47.102:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:0d2e1f77f8531ecfbccdace3dd0f78507fe31c6ce1da02cecf5f35788a8ec613

3.6、Node节点加入集群Node节点都要操作

在待添加的worker节点上执行

# worker01 节点
[root@localhost containerd]# kubeadm join 192.168.47.102:6443 --token abcdef.0123456789abcdef \
>         --discovery-token-ca-cert-hash sha256:0d2e1f77f8531ecfbccdace3dd0f78507fe31c6ce1da02cecf5f35788a8ec613
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

集群当前状态

需要对notready的节点安装部署网络插件即可。

四、部署网络插件（CNI）

4.1、拉取必要镜像

所有节点都需要拉取以下两个镜像
在部署网络插件的过程中，实测是可能会有两个镜像拉取不来下，所以手动拉取一下，不过要使用魔法，拉取不到了评论或私信（开源免费）

以下是拉取镜像的命令，所有节点都需要有这两个镜像

crictl pull docker.io/flannel/flannel-cni-plugin:v1.1.2
crictl pull docker.io/flannel/flannel:v0.21.5

这里如果拉取失败，我们可以使用在有dokcer环境的机器上先将镜像拉下来，然后传到当前机器上进行解压后执行后续的相关操作。

[root@localhost ~]# ctr -n k8s.io images import flannel.tar
unpacking docker.io/flannel/flannel:v0.21.5 (sha256:6f2d991efb758c5530e7de90761dfb29637b7604a807d431312e20189e09f9e6)...done
[root@localhost ~]# ctr -n k8s.io images import flannel-cni-plugin.tar
unpacking docker.io/flannel/flannel-cni-plugin:v1.1.2 (sha256:539d3bf046c8581557f0747dbad9d3b78a4de112d3c0bf9d291651593060fc9f)...done

4.2、部署网络插件

master节点操作即可

#下载flannel.yaml文件
wget https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml

[root@master ~]# kubectl apply -f kube-flannel.yaml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

4.2、查看节点状态

[root@admin01 ~]# kubectl  get nodes
NAME       STATUS   ROLES           AGE   VERSION
master     Ready    control-plane   20m   v1.27.0
worker01   Ready    <none>          18m   v1.27.0

4.3、查看组件状态

[root@admin01 ~]# kubectl  get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE                         ERROR
etcd-0               Healthy   {"health":"true","reason":""}
controller-manager   Healthy   ok
scheduler            Healthy   ok

4.4、查看所有Pod状态

[root@admin01 ~]# kubectl  get pod -A
NAMESPACE      NAME                             READY   STATUS    RESTARTS   AGE
kube-flannel   kube-flannel-ds-bgnfw            1/1     Running   0          6m26s
kube-flannel   kube-flannel-ds-bmdr8            1/1     Running   0          2m36s
kube-system    coredns-5d78c9869d-9vmpk         1/1     Running   0          21m
kube-system    coredns-5d78c9869d-wzb8q         1/1     Running   0          21m
kube-system    etcd-master                      1/1     Running   0          21m
kube-system    kube-apiserver-master            1/1     Running   0          21m
kube-system    kube-controller-manager-master   1/1     Running   0          6m56s
kube-system    kube-proxy-shnbq                 1/1     Running   0          21m
kube-system    kube-proxy-trzth                 1/1     Running   0          19m
kube-system    kube-scheduler-master            1/1     Running   0          21m

4.5 解决异常pod状态

查看异常pod的描述状态发现

部署flannel网络插件时发现flannel一直处于CrashLoopBackOff状态，查看日志提示没有分配cidr

解决

vim /etc/kubernetes/manifests/kube-controller-manager.yaml

增加参数：

--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16

systemctl restart kubelet

五、Containerd基本操作

5.1、Crictl镜像管理

# 下载镜像
[root@master ~]# crictl pull nginx:latest

# 查看所有镜像
[root@master ~]# crictl images

# 查看镜像详细信息
[root@master ~]# crictl inspecti nginx:latest

# 删除镜像
[root@master ~]# crictl rmi nginx:latest

5.2、Crictl容器管理

# 列出运行中的容器，查看所有容器加-a选项
[root@master ~]# crictl ps

# 查看容器的详细信息
[root@master ~]# crictl inspect 07376dc402441

# 启动容器
[root@master ~]# crictl start 07376dc402441

# 停止容器
[root@master ~]# crictl stop 07376dc402441

# 删除容器
[root@master ~]# crictl rm 07376dc402441

# 打印日志
crictl logs <container-id 或 container-name>

# 进入容器执行命令
crictl exec -it <container-id 或 container-name> sh

5.3、Crictl Pod管理

# 打印所有 Pod
[root@node2 ~]# crictl pods
# 根据pod名称打印pod信息
[root@master ~]# crictl pods --name kube-proxy-trnjn

# 根据标签打印 Pod，crictl pods --label key=value
[root@master ~]# crictl pods --label k8s-app=kube-proxy

【云原生】最新版Kubernetes集群基于Containerd部署

文章目录

Kubernetes集群基于Containerd部署

资源列表

基础环境

192.168.47.102机器执行

192.168.47.103机器执行

一、基础环境准备

1.1、关闭Swap分区

1.2、添加hosts解析

1.3、桥接的IPv4流量传递给iptables的链

二、准备Containerd容器运行时

2.1、安装Containerd

添加 docker 源，containerd也在docker源内的

快速建立 yum 缓存

安装containerd

列出所有containerd版本

2.2、配置Containerd

2.3、启动Containerd

三、部署Kubernetes集群

3.1、安装Kubeadm工具

快速建立yum缓存

安装 kubectl：命令行管理工具、kubeadm：安装K8S集群工具、kubelet管理容器工具

切记这个时候不要启动，只需要设置为开机自启

3.2、配置crictl工具

所有节点都要操作

3.3、配置Containerd镜像加速器

3.4、测试crictl工具是否可用

3.5、初始化Master节点

3.6、Node节点加入集群Node节点都要操作

四、部署网络插件（CNI）

4.1、拉取必要镜像

以下是拉取镜像的命令，所有节点都需要有这两个镜像

4.2、部署网络插件

4.2、查看节点状态

4.3、查看组件状态

4.4、查看所有Pod状态

4.5 解决异常pod状态

五、Containerd基本操作

5.1、Crictl镜像管理

5.2、Crictl容器管理

5.3、Crictl Pod管理