主机规划
|---------------|------------------|------------|
| IP | hostname | 备注 |
| 192.168.2.101 | k8s-master | 主节点 |
| 192.168.2.102 | k8s-node01 | work节点1 |
| 192.168.2.103 | k8s-node02 | work节点2 |
| 192.168.2.20 | harbor.muhuo.com | harbor镜像仓库 |
版本信息
|-------------------------|-------------|
| containerd | 1.7.28 |
| kubeadm、kubelet、kubectl | 1.28.15-1.1 |
| kube-apiserver | v1.28.15 |
| kube-controller-manager | v1.28.15 |
| kube-scheduler | v1.28.15 |
| kube-proxy | v1.28.15 |
| etcd | 3.5.15-0 |
| coredns | v1.10.1 |
| calico | v3.28.3 |
环境准备：
每个节点分别设置对应主机名
hostnamectl set-hostname k8s-master
hostnamectl set-hostname k8s-node01
hostnamectl set-hostname k8s-node02
设置主机名 /etc/hosts
192.168.2.20 harbor.muhuo.com
192.168.2.101 k8s-master
192.168.2.102 k8s-node01
192.168.2.103 k8s-node02
关闭交换分区
swapoff -a && sysctl -w vm.swappiness=0
sed -ri 's/.*swap.*/#&/' /etc/fstab
安装时间同步服务
apt-get -y install chrony
chronyc sources -v
timedatectl set-timezone Asia/Shanghai
禁用防火墙服务
ufw disable
ufw status
在所有节点上开启IP转发
cat <<EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
#开启ipvs
apt install -y ipset ipvsadm
配置加载模块
cat > /etc/modules-load.d/ipvs.conf << EOF
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
临时加载
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
开机加载配置，将ipvs相关模块加入配置文件中
cat >> /etc/modules <<EOF
ip_vs_sh
ip_vs_wrr
ip_vs_rr
ip_vs
nf_conntrack
EOF
在所有节点上添加 Kubernetes 的阿里云源（我这里没用到）：
apt-get update && apt-get install -y apt-transport-https
curl -fsSL https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/Release.key |
gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/ /" |
tee /etc/apt/sources.list.d/kubernetes.list
apt-get update
kubeadm、containerd安装：
安装 kubeadm、kubelet、kubectl
apt install -y kubeadm=1.28.15-1.1 kubelet=1.28.15-1.1 kubectl=1.28.15-1.1
安装containerd
apt install -y containerd
生成containerd默认配置文件
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml
修改containerd配置 指向harbor仓库
sed -i 's#sandbox_image = ".*"#sandbox_image = "harbor.muhuo.com/k8s-image/pause:3.9"#' /etc/containerd/config.toml
sed -i 's#SystemdCgroup = false#SystemdCgroup = true#' /etc/containerd/config.toml
创建 containerd 认证配置
sudo mkdir -p /etc/containerd/certs.d/harbor.muhuo.com
配置harbor私有仓库全局认证
sudo cat > /etc/containerd/certs.d/harbor.muhuo.com/hosts.toml <<EOF
server = "https://harbor.muhuo.com"
host."https://harbor.muhuo.com"
capabilities = ["pull", "resolve"]
skip_verify = true # 如果是自签名证书
host."https://harbor.muhuo.com".header
Authorization = ["Basic $(echo -n 'kubernetes:password' | base64)"]
EOF
上传harbor证书文件到/etc/containerd/certs.d/harbor.muhuo.com/
cat > /etc/containerd/certs.d/harbor.muhuo.com/hosts.toml <<EOF
server = "https://harbor.muhuo.com"
host."https://harbor.muhuo.com"
capabilities = ["pull", "resolve"]
ca = "/etc/containerd/certs.d/harbor.muhuo.com/harbor.crt"
EOF
重启 containerd
sudo systemctl restart containerd
设置crictl
cat > /etc/crictl.yaml << EOF
runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
timeout: 10
debug: false
EOF
查看需要的镜像列表
kubeadm config images list
集群搭建
初始化集群
kubeadm init \
--apiserver-advertise-address=192.168.2.101 \
--image-repository harbor.muhuo.com/k8s-image \
--kubernetes-version v1.28.15 \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.96.0.0/16
根据提示需要创建相关命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown (id -u):(id -g) $HOME/.kube/config
分别再node01和node02执行以下命令加入集群
kubeadm join 192.168.2.101:6443 --token 88xry2.r3b1ngvb2ytr1adn \
--discovery-token-ca-cert-hash sha256:33ade49f640c3e1d1ad446bfd6cc143637693863341314b4b82039db660ff97b
集群构建成功
其他配置修改
kubectl命令自动补全
kubectl completion bash > ~/.kube/completion.bash.inc
echo source '$HOME/.kube/completion.bash.inc' >> ~/.bashrc
source ~/.bashrc
修改mode为ipvs模式（ipvs相比iptables性能更高）
kubectl edit cm kube-proxy -n kube-system
删除现有kube-proxy pod实现自动更新
kubectl delete pod -n kube-system -l k8s-app=kube-proxy
设置对应节点的标签
kubectl label node k8s-master node-role.kubernetes.io/master='master'
kubectl label node k8s-node01 node-role.kubernetes.io/worker='node'
kubectl label node k8s-node02 node-role.kubernetes.io/worker='node'
网络插件安装
下载该文件到本地，修改文件中镜像地址为本地镜像仓库地址
https://raw.githubusercontent.com/projectcalico/calico/v3.28.3/manifests/calico.yaml
kubectl apply -f calico.yaml
至此Kubernetes集群已经完整安装完成。