一、集群节点规划
注意:Kubernetes 1.36 已经彻底移除 docker shim,只能使用 containerd/crun,不再支持 docker 直接作为 runtime。
|------------|----------------------------------------|
| 操作系统 | Ubuntu 24.04.4 LTS 内核6.8.0-138-generic |
| Containerd | v2.2.1 |
| k8s | v1.36.4 |
| kubeadm | v1.36 |
[软件环境]
|-------|------|--------------|-----|-----|
| 主机 | 角色 | IP | CPU | 内存 |
| node1 | 控制平面 | 192.168.5.10 | 16C | 40G |
| node2 | 工作节点 | 192.168.5.11 | 16C | 40G |
| node3 | 工作节点 | 192.168.5.12 | 16C | 40G |
[节点配置]
注意:
- 控制平面至少 2 核、4 GB 内存
- 工作节点至少 2 核、4 GB 内存
- 所有节点使用固定 IP
- 节点之间可以互相访问
- 不要使用公网 IP 作为集群节点通信地址
- 主机名不能重复
- 普通用户 + sudo(官方推荐)方式安装
二、设置主机名和 hosts
分别在每个节点设置对应主机名,例如:
sudo hostnamectl set-hostname node1
所有节点配置/etc/hosts:
sudo tee -a /etc/hosts > /dev/null <<EOF
192.168.5.10 node1
192.168.5.11 node2
192.168.5.12 node3
EOF
验证:
root@node1:~# hostname
node1
三、更新系统并安装基础工具
bash
sudo apt update
sudo apt upgrade -y
sudo apt install -y \
apt-transport-https \
ca-certificates \
curl \
gpg \
chrony \
socat \
conntrack \
ipset \
ebtables \
ethtool \
lsof
四、保持时间同步
bash
sudo systemctl enable --now chrony
timedatectl status
设置时区
bash
# 设置时区为上海
sudo timedatectl set-timezone Asia/Shanghai
# 验证
timedatectl
五、关闭 Swap
Kubernetes 默认检测到 Swap 时会导致 kubelet 无法正常启动。官方建议关闭 Swap,或者额外配置 kubelet 允许 Swap。生产环境建议直接关闭。
临时关闭:
bash
sudo swapoff -a
永久关闭:
bash
sudo sed -i.bak '/[[:space:]]swap[[:space:]]/ s/^/#/' /etc/fstab
检查:
bash
free -h
swapon --show
六、配置内核模块和网络参数
加载内核模块:
bash
sudo tee /etc/modules-load.d/k8s.conf > /dev/null <<EOF
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
配置网络参数:
bash
sudo tee /etc/sysctl.d/k8s.conf > /dev/null <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
检查:
bash
sysctl net.ipv4.ip_forward
lsmod | grep br_netfilter
预期:
bash
net.ipv4.ip_forward = 1
七、安装并配置 containerd
bash
sudo apt install -y containerd
检查:
bash
pan@node1:~$ sudo ctr version
Client:
Version: 2.2.1
Revision:
Go version: go1.24.4
Server:
Version: 2.2.1
Revision:
UUID: 9844b409-4c1c-4f55-b0d4-164c16ea759e
生成默认配置:
bash
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
启用 systemd cgroup:
bash
sudo sed -i \
's/SystemdCgroup = false/SystemdCgroup = true/' \
/etc/containerd/config.toml
确认 CRI 插件没有被禁用:
bash
sudo grep -n "disabled_plugins\|SystemdCgroup" /etc/containerd/config.toml
预期:
bash
disabled_plugins = ["cri"]
重启并设置开机启动:
bash
sudo systemctl restart containerd
sudo systemctl enable containerd
sudo systemctl status containerd --no-pager
八、安装 kubeadm、kubelet、kubectl
生产环境建议所有节点使用完全相同的 Kubernetes 版本,本文主要介绍Kubernetes v1.36的安装。
注意:Kubernetes 官方已弃用旧的 apt.kubernetes.io 仓库,应使用新的 pkgs.k8s.io 仓库。
bash
sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key \
| sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' \
| sudo tee /etc/apt/sources.list.d/kubernetes.list
安装组件:
bash
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
检查版本:
bash
kubeadm version
kubelet --version
kubectl version --client
锁定版本,避免系统升级时自动升级 Kubernetes:
bash
sudo apt-mark hold kubelet kubeadm kubectl
启用 kubelet:
bash
sudo systemctl enable kubelet
九、关闭防火墙
Ubuntu 默认防火墙工具是 ufw,底层是 iptables/nftables。
k8s 测试环境可以关闭;生产环境建议不要完全关闭,而是放行集群端口。
bash
sudo ufw disable
sudo systemctl stop ufw
sudo systemctl disable ufw
十、禁用IPV6
bash
sudo tee /etc/sysctl.d/99-disable-ipv6.conf > /dev/null <<EOF
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
EOF
sudo sysctl --system
验证:
bash
cat /proc/sys/net/ipv6/conf/all/disable_ipv6
输出 1 表示已禁用。
重启相关服务:
bash
sudo systemctl restart containerd
sudo systemctl restart kubelet
十一、使用配置文件进行初始化控制平面
相比长命令,配置文件更容易保存和后续复用。
创建配置文件:
bash
sudo mkdir -p /etc/kubernetes
sudo vim /etc/kubernetes/kubeadm-config.yaml
写入:
bash
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.5.10
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 192.168.5.10
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.36.4
imageRepository: registry.aliyuncs.com/google_containers
controlPlaneEndpoint: "192.168.5.10:6443"
networking:
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
dnsDomain: cluster.local
注意:
advertiseAddress填控制平面节点 IPnode-ip填本机实际用于集群通信的 IPcontrolPlaneEndpoint是其他节点访问 API Server 的地址podSubnet必须与后续 CNI 网络插件配置一致- 如果未来计划做高可用,
controlPlaneEndpoint应填写负载均衡器或稳定 DNS,而不是单个节点 IP
查看需要拉取的镜像
bash
pan@node1:~$ sudo kubeadm config images list \
--config=/etc/kubernetes/kubeadm-config.yaml
registry.aliyuncs.com/google_containers/kube-apiserver:v1.36.4
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.36.4
registry.aliyuncs.com/google_containers/kube-scheduler:v1.36.4
registry.aliyuncs.com/google_containers/kube-proxy:v1.36.4
registry.aliyuncs.com/google_containers/coredns:v1.14.2
registry.aliyuncs.com/google_containers/pause:3.10.2
registry.aliyuncs.com/google_containers/etcd:3.6.8-0
先执行预检查:
bash
sudo kubeadm init \
--config=/etc/kubernetes/kubeadm-config.yaml \
--dry-run
执行初始化:
bash
sudo kubeadm init \
--config=/etc/kubernetes/kubeadm-config.yaml
如果初始化失败,查看日志:
bash
journalctl -u kubelet -n 100 --no-pager
我第一次安装失败,查找日志找到了报错信息:
failed to pull image "registry.k8s.io/pause:3.10.1"
先测试一下是否可拉取镜像
bash
sudo crictl pull registry.k8s.io/pause:3.10.1
ctr: failed to resolve image: failed to do request: Head "https://europe-west3-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.10.1?rid=0f7a60629231ae3195f59506e39f5118": dial tcp 74.125.142.82:443: i/o timeout
说明网络有问题无法下载,可以修改为国内镜像地址
bash
sudo sed -i 's#registry.k8s.io/pause:3.10.1#registry.aliyuncs.com/google_containers/pause:3.10.1#' /etc/containerd/config.toml
重启 containerd
bash
sudo systemctl restart containerd
测试拉取:
bash
apt install -y cri-tools
crictl pull registry.aliyuncs.com/google_containers/pause:3.10.1
crictl images | grep pause
执行清理后重新初始化
bash
sudo kubeadm reset -f
sudo rm -rf /etc/cni/net.d
sudo rm -rf /var/lib/cni
sudo systemctl restart containerd
sudo systemctl restart kubelet
初始化成功后,终端会输出类似下面的 Worker 加入命令:
bash
kubeadm join 192.168.5.10:6443 \
--token <token> \
--discovery-token-ca-cert-hash sha256:<hash>
需要保存下命令。
十二、配置 kubectl
普通用户:
bash
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
root用户:
bash
export KUBECONFIG=/etc/kubernetes/admin.conf
十三、Worker 节点加入集群
bash
sudo kubeadm join 192.168.5.10:6443 --token xxx \
--discovery-token-ca-cert-hash xxx
十四、安装 Calico 网络插件
建议安装 Calico Open Source v3.32.1。Calico 官方已测试支持 Kubernetes 1.36。
安装 Tigera Operator 和 CRD
bash
kubectl create -f \
https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/v1_crd_projectcalico_org.yaml
kubectl create -f \
https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/tigera-operator.yaml
检查 Operator:
bash
kubectl get pods -n tigera-operator
下载并修改 Calico 配置
bash
curl -LO \
https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/custom-resources.yaml
查看配置中的 Pod 网段:
bash
grep -n "cidr:" custom-resources.yaml
修改为:
bash
sed -i 's#192.168.0.0/16#10.244.0.0/16#g' \
custom-resources.yaml
创建 Calico 资源
bash
kubectl create -f custom-resources.yaml
查看集群状态:
bash
kubectl get tigerastatus
验证节点状态
bash
kubectl get nodes -o wide
移除控制平面污点
bash
kubectl taint nodes node1 node-role.kubernetes.io/control-plane:NoSchedule-
如果遇到calico node event报错:
calico/node is not ready: BIRD is not ready: BGP not established with 192.168.5.11,192.168.5.12
修改Tigera calico网络配置:
bash
spec:
# Configures Calico networking.
calicoNetwork:
nodeAddressAutodetectionV4:
firstFound: false
interface: "enp1s0"
应用并重启 calico‑node 使变更生效:
bash
kubectl apply -f custom-resources.yaml
kubectl rollout restart daemonset/calico-node -n calico-system
验证:
bash
kubectl get pods -A -w|grep calico-node
十五、测试集群
创建测试 Deployment:
bash
kubectl create deployment nginx --image=docker.m.daocloud.io/library/nginx:1.25
kubectl expose deployment nginx --port=80 --type=NodePort
查看服务:
bash
kubectl get pods -o wide
kubectl get svc nginx
访问服务:
bash
http://192.168.5.10:<nodeport>