集群分布
适用环境：3节点集群（1 Master + 2 Worker）、Rocky Linux 10.1
集群规划：
主机名 IP地址 角色 核心组件 k8s-master01 192.168.24.11 Master apiserver/controller-manager/scheduler/etcd/kubelet/kube-proxy/cri-dockerd/calico k8s-node01 192.168.24.12 Worker kubelet/kube-proxy/cri-dockerd k8s-node02 192.168.24.13 Worker kubelet/kube-proxy/cri-dockerd
一、系统初始化（所有节点执行）
1.1 本地仓库配置（离线环境必做）
你之前已经挂载了Rocky 10.1 ISO作为本地仓库，确认配置正确：
# 确认ISO已挂载
mount -a
df -h /mnt/iso
# 备份原有repo，只保留本地仓库
cd /etc/yum.repos.d/
mkdir backup
mv rocky*.repo backup/
# 如果有自定义本地repo，确保enabled=11.2 关闭防火墙&SELinux
systemctl disable --now firewalld
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
setenforce 01.3 关闭Swap（K8s硬性要求）
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab
free -m # 确认Swap全为01.4 安装基础依赖
dnf install -y net-tools nfs-utils vim wget curl bash-completion device-mapper-persistent-data psmisc tree openssh openssh-server ipvsadm
# 加载bash补全（不用退出shell）
source /etc/profile.d/bash_completion.sh1.5 系统参数优化
# 最大连接数
cat >> /etc/security/limits.conf <<EOF
* soft nofile 655350
* hard nofile 655350
* soft nproc 655350
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited
EOF
# 内核参数（K8s网络要求）
cat >> /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
cat >> /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1 #主要
EOF
sysctl --system or sysctl -p
# 时间同步
systemctl enable --now chronyd
chronyc sources # 确认时间源正常
timedatectl set-timezone Asia/Shanghai二、节点基础配置（所有节点执行）
2.1 修改主机名
# master节点
hostnamectl hostname k8s-master01
# node01节点
hostnamectl hostname k8s-node01
# node02节点
hostnamectl hostname k8s-node022.2 配置静态IP（Rocky10.1默认网卡为ens160）
# 以node01为例，其他节点替换IP即可
nmcli c m ens160 ipv4.method manual \
ipv4.addresses 192.168.24.12/24 \
ipv4.dns "223.5.5.5 8.8.8.8" \
ipv4.gateway 192.168.24.2 \
connection.autoconnect yes
nmcli c up ens1602.3 配置主机映射（所有节点一致）
cat > /etc/hosts <<EOF
192.168.24.11 k8s-master01 m1
192.168.24.12 k8s-node01 n1
192.168.24.13 k8s-node02 n2
EOF三、容器运行时安装（所有节点执行，选cri-dockerd）
⚠️ 不要同时安装containerd和cri-dockerd，二选一即可
3.1 安装cri-dockerd
# 提前下载好cri-dockerd-0.3.17.amd64.tgz（离线环境提前拷贝到节点）
tar -xf cri-dockerd-0.3.17.amd64.tgz
cp cri-dockerd/cri-dockerd /usr/bin/
chmod +x /usr/bin/cri-dockerd3.2 配置cri-docker服务
# cri-docker.service
cat > /usr/lib/systemd/system/cri-docker.service <<EOF
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.10
ExecReload=/bin/kill -s HUP \$MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF
# cri-docker.socket
cat > /usr/lib/systemd/system/cri-docker.socket <<EOF
[Unit]
Description=CRI Docker socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target
EOF3.3 启动cri-docker
systemctl daemon-reload
systemctl enable --now cri-docker
systemctl is-active cri-docker # 确认输出active✅ 【必做修正·踩坑重点】
K8s 1.36的kubelet默认会找registry.k8s.io/pause:3.10，哪怕service里配了阿里云的镜像，所有节点必须额外打这个tag：
docker tag registry.aliyuncs.com/google_containers/pause:3.10 registry.k8s.io/pause:3.10
docker images | grep pause # 确认两个tag都存在四、Docker安装（所有节点执行，离线环境提前准备rpm包）
这里按照docker步骤省略，前面笔记有详细docker安装步骤
4.1 配置Docker（适配K8s）
# 用cat命令直接写入配置（无需vim手动编辑，避免格式错误）
[root@k8s-master01 ~]# cat /etc/docker/daemon.json
{
"default-ipc-mode": "shareable",
"data-root": "/data/docker",
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "50"
},
"insecure-registries": ["https://hb.reg.com"], #配置Harbor仓库的网址
"registry-mirrors": [
"https://d8jozvml.mirror.aliyuncs.com",
"https://docker.1ms.run",
"https://func.ink",
"https://proxy.1panel.live",
"https://docker-0.unsee.tech",
"https://docker.zhai.cm",
"https://a.ussh.net",
"https://docker.melikeme.cn",
"https://lispy.org",
"https://docker.hlmirror.com",
"https://docker.xiaogenban1993.com",
"https://docker.1panel.top",
"https://docker.kejilion.pro",
"https://dockerpull.cn",
"https://docker.xuanyuan.me",
"https://docker.anye.in",
"https://hub.fast360.xyz"
]
}4.2 启动Docker
systemctl enable --now docker
docker version # 确认client和server都正常五、K8s组件安装（所有节点执行）
5.1 配置K8s阿里云仓库（适配1.36版本）
cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.36/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.36/rpm/repodata/repomd.xml.key
EOF5.2 安装kubelet/kubeadm/kubectl
dnf makecache
dnf install -y kubelet kubeadm kubectl
systemctl enable --now kubelet⚠️ 此时kubelet会反复重启是正常现象 ，因为还没执行kubeadm init，不用处理。
5.3 镜像准备
Master节点：导入所有K8s镜像
提前把k8s-images_v1.36.3.tar.gz拷贝到master节点：
我这个包装有需要的镜像
docker load < k8s-images_v1.36.3.tar.gz
# 修正镜像tag（适配阿里云仓库）
docker tag registry.k8s.io/coredns/coredns:v1.14.2 registry.aliyuncs.com/google_containers/coredns:v1.14.2
docker tag registry.k8s.io/etcd:3.6.11-0 registry.aliyuncs.com/google_containers/etcd:3.6.11-0
# 确认所有镜像都存在
kubeadm config images list --image-repository registry.aliyuncs.com/google_containers如若没有，先查看需要的镜像在拉取
[root@k8s-master01 ~]# kubeadm config images list
registry.k8s.io/kube-apiserver:v1.36.3
registry.k8s.io/kube-controller-manager:v1.36.3
registry.k8s.io/kube-scheduler:v1.36.3
registry.k8s.io/kube-proxy:v1.36.3
registry.k8s.io/coredns/coredns:v1.14.2
registry.k8s.io/pause:3.10.2
registry.k8s.io/etcd:3.6.8-0
# 获取镜像文件
images=$(kubeadm config images list --kubernetes-version=1.36.3 | awk -F "/" '{print $NF}')
for i in ${images}
do
docker pull registry.aliyuncs.com/google_containers/$i
docker tag registry.aliyuncs.com/google_containers/$i harbor.registry.com/k8s/$i
docker push harbor.registry.com/k8s/$i
docker rmi registry.aliyuncs.com/google_containers/$i
doneWorker节点：只导入3个必要镜像
提前把k8s-images_v1.36.3.tar.gz拷贝到node01/node02：
docker load < k8s-images_v1.36.3.tar.gz
# 只保留pause、coredns、kube-proxy三个镜像即可
docker tag registry.k8s.io/pause:3.10.2 registry.k8s.io/pause:3.10
docker tag registry.k8s.io/pause:3.10.2 registry.aliyuncs.com/google_containers/pause:3.10
docker tag registry.k8s.io/coredns/coredns:v1.14.2 registry.aliyuncs.com/google_containers/coredns:v1.14.2六、Master节点初始化（仅在k8s-master01执行）
6.1 执行初始化命令
⚠️ 必须加--cri-socket参数，否则会报错
kubeadm init \
--apiserver-advertise-address=192.168.24.11 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version=1.36.3 \
--service-cidr=10.10.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--cri-socket unix:///var/run/cri-dockerd.sock这次最后面会首次产生join命令在最后面
6.2 初始化后配置
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config6.3 验证Master状态
kubectl get nodes # 此时master为NotReady，正常，还没装网络插件
kubectl get pods -A # coredns应为Pending，正常七、Worker节点加入集群（仅在node01/node02执行）
7.1 先清理旧残留（如果之前执行过join失败）
kubeadm reset --cri-socket=unix:///var/run/cri-dockerd.sock -f7.2 生成join命令（Master节点执行，token24小时过期）
kubeadm token create --print-join-command
# 输出类似：
# kubeadm join 192.168.24.11:6443 --token xxxx --discovery-token-ca-cert-hash sha256:xxxx7.3 执行join（必须加--cri-socket参数）
kubeadm join 192.168.24.11:6443 \
--token xxxx \
--discovery-token-ca-cert-hash sha256:xxxx \
--cri-socket=unix:///var/run/cri-dockerd.sock成功例子：
[root@k8s-node01 ~]# kubeadm join 192.168.24.11:6443 \
--token 8lgvi9.17nbgxhas9odzbsn \
--discovery-token-ca-cert-hash sha256:9e7715488e413ed6279a782ea62d502b2c174c6947c3764df1ac152ffeb56317 \
--cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[preflight] Reading configuration from the "kubeadm-config" ConfigMap in namespace "kube-system"...
[preflight] Use 'kubeadm init phase upload-config kubeadm --config your-config-file' to re-upload it.
W0727 14:46:12.692581 2720 utils.go:69] The recommended value for "bindAddress" in "KubeProxyConfiguration" is: ::; the provided value is: 0.0.0.0
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 501.939882ms
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.八、Calico网络插件部署（仅在Master节点执行）
8.1 下载Calico配置文件
提前下载calico.yaml（离线环境提前拷贝到master）：
50节点以下
curl -O https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico.yaml50节点+
# 下载带Typha的Manifest
curl -fsSL https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico-typha.yaml -o calico-typha.yaml8.2 修改Calico配置
vim里面用:/去跳转查询
vim calico.yaml
# 找到以下配置，修改/取消注释：
# 1. 关闭IPIP和VXLAN，用BGP模式
- name: CALICO_IPV4POOL_IPIP
value: "off"
- name: CALICO_IPV4POOL_VXLAN
value: "Never"
# 2. 修改Pod网段为初始化时指定的10.244.0.0/16
- name: CALICO_IPV4POOL_CIDR
value: "10.244.0.0/16"8.3 导入Calico镜像（所有节点执行）
提前把calico-images_v3.32.1.tar.gz拷贝到所有节点：
calico镜像确实国内比较难下载
docker load < calico-images_v3.32.1.tar.gz想要知道要用什么镜像（通过上面下载的calico配置文件）：
50节点以下：只有3个镜像
[root@k8s-master01 ~]# grep 'image:' calico.yaml | sort -u
image: quay.io/calico/cni:v3.32.1
image: quay.io/calico/kube-controllers:v3.32.1
image: quay.io/calico/node:v3.32.1
50节点+，4个镜像，同样的方法查询
quay.io/calico/cni:v3.32.1 \
quay.io/calico/node:v3.32.1 \
quay.io/calico/kube-controllers:v3.32.1 \
quay.io/calico/typha:v3.32.18.4 部署Calico
kubectl apply -f calico.yaml8.5 验证集群状态
等待2-3分钟，执行以下命令，所有组件均为Running/Ready即为成功：
[root@k8s-master01 ~]# kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master01 Ready control-plane 4h31m v1.36.3 192.168.24.11 <none> Rocky Linux 10.2 (Red Quartz) 6.12.0-211.16.1.el10_2.0.1.x86_64 (amd64) docker://29.6.2
k8s-node01 Ready <none> 4h8m v1.36.3 192.168.24.12 <none> Rocky Linux 10.2 (Red Quartz) 6.12.0-211.16.1.el10_2.0.1.x86_64 (amd64) docker://29.6.2
k8s-node02 Ready <none> 4h8m v1.36.3 192.168.24.13 <none> Rocky Linux 10.2 (Red Quartz) 6.12.0-211.16.1.el10_2.0.1.x86_64 (amd64) docker://29.6.2
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready control-plane 4h32m v1.36.3
k8s-node01 Ready <none> 4h8m v1.36.3
k8s-node02 Ready <none> 4h8m v1.36.3
kubectl get pods -A
# 所有Pod均为Running，无Pending/Init状态
[root@k8s-master01 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-58f68d56d9-6sf8g 1/1 Running 0 61m
kube-system calico-node-nnnc9 1/1 Running 0 61m
kube-system calico-node-rfdzz 1/1 Running 3 (58m ago) 61m
kube-system calico-node-sfs7f 1/1 Running 0 61m
kube-system coredns-589f44dc88-fh5mn 1/1 Running 0 4h31m
kube-system coredns-589f44dc88-q28hr 1/1 Running 0 4h31m
kube-system etcd-k8s-master01 1/1 Running 1 (68m ago) 4h31m
kube-system kube-apiserver-k8s-master01 1/1 Running 1 (68m ago) 4h31m
kube-system kube-controller-manager-k8s-master01 1/1 Running 1 (68m ago) 4h31m
kube-system kube-proxy-89t5s 1/1 Running 1 (68m ago) 4h31m
kube-system kube-proxy-gk5ql 1/1 Running 0 4h8m
kube-system kube-proxy-h86dq 1/1 Running 0 4h8m
kube-system kube-scheduler-k8s-master01 1/1 Running 1 (68m ago) 4h31m九、常见错误排查
| 报错现象 | 原因 | 解决方案 |
|---|---|---|
join时报found multiple CRI endpoints |
未指定cri-socket | join命令加--cri-socket=unix:///var/run/cri-dockerd.sock |
calico-node卡Init:0/3 |
worker节点缺pause镜像/calico镜像 | 1. worker节点导入pause镜像并打registry.k8s.io/pause:3.10tag 2. 所有节点导入calico镜像 3. 重启kubelet |
yaml解析报错did not find expected '-' indicator |
sed修改calico.yaml破坏了缩进 | 手动vim修改calico.yaml，不要批量替换 |
worker节点执行kubectl报connection refused |
worker节点无kubeconfig | 到master节点执行kubectl，或拷贝/etc/kubernetes/admin.conf到worker的~/.kube/config |
kubelet报failed to pull pause:3.10 |
未打kubelet默认找的pause tag | 所有节点执行docker tag registry.k8s.io/pause:3.10.2 registry.k8s.io/pause:3.10 |
十、集群验证（可选）
部署测试nginx服务，验证跨节点通信：
kubectl create deployment nginx --image=nginx:alpine --replicas=3
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pods -o wide # 确认Pod分布在3个节点
curl http://任意节点IP:NodePort # 能访问nginx页面即为集群完全正常到此Rocky10.1 + K8s 1.36.3集群已完全就绪，建议给所有节点打快照，方便后续回滚。