快速搭建 K8s 集群
| 角色 | ip |
|---|---|
| k8s-master-01 | 192.168.111.170 |
| k8s-node-01 | 192.168.111.171 |
| k8s-node-02 | 192.168.111.172 |
服务器需要连接互联网下载镜像
| 软件 | 版本 |
|---|---|
| Docker | 24.0.0（CE） |
| Kubernetes | 1.28 |
初始化配置
关闭防火墙
sh
systemctl stop firewalld && systemctl disable firewalld关闭Selinux
sh
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0关闭Swap
sh
sed -ri 's/.*swap.*/#&/' /etc/fstab
swapoff -a根据规划设置主机名
sh
hostnamectl set-hostname k8s-master-01
hostnamectl set-hostname k8s-node-01
hostnamectl set-hostname k8s-node-02网络桥段
sh
vi /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_forward=1
net.ipv4.ip_forward_use_pmtu = 0
# 生效命令
sysctl --system
# 查看效果
sysctl -a|grep "ip_forward"确保网络桥接的数据包经过Iptables处理，防止网络丢包
sh
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system # 生效同步时间
sh
# 安装软件
yum -y install ntpdate
# 向阿里云服务器同步时间
ntpdate time1.aliyun.com
# 删除本地时间并设置时区为上海
rm -rf /etc/localtime && ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# 查看时间
date -R || date开启 IPVS
sh
yum -y install ipset ipvsdm
cat > /etc/sysconfig/modules/ipvs.modules << EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
# 赋予权限并执行
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules &&lsmod | grep -e ip_vs -e nf_conntrack_ipv4
# 重启电脑，检查是否生效
reboot命令补全
sh
yum -y install bash-completion bash-completion-extras
source /etc/profile.d/bash_completion.sh配置 HOSTS
sh
cat <<EOF >>/etc/hosts
192.168.111.170 k8s-master-01
192.168.111.171 k8s-node-01
192.168.111.172 k8s-node-02
EOF
cat <<EOF >>/etc/hosts
192.168.111.173 k8s-tool-01 harbor.liuyuncen.com
EOF安装docker
下载源
sh
yum install -y wget
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo安装 docker
sh
# yum list docker-ce --showduplicates | sort -r
yum -y install docker-ce-24.0.0
systemctl enable docker && systemctl start docker设置Cgroup驱动
sh
cat > /etc/docker/daemon.json << EOF
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
systemctl daemon-reload && systemctl restart docker
# 查看设置状态
docker info安装 cri-docker 驱动 （Docker与Kubernetes通信的中间程序）：
sh
# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.2/cri-dockerd-0.3.2-3.el7.x86_64.rpm
rpm -ivh cri-dockerd-0.3.2-3.el7.x86_64.rpm指定依赖镜像地址为国内镜像地址：
sh
vim /usr/lib/systemd/system/cri-docker.service
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9
systemctl daemon-reload
systemctl enable cri-docker && systemctl start cri-docker部署 K8s 集群
添加阿里云 yum 源
sh
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF安装kubeadm 官方提供的集群搭建工具，kubelet 守护进程 和kubectl 管理集群工具（实际在 master 安装即可）
sh
yum install -y kubelet-1.28.0 kubeadm-1.28.0 kubectl-1.28.0
systemctl enable kubelet
# 这里只是设置开机启动，直接起也起不来提前把所有镜像都拉下来
sh
docker load -i calico.v3.25.1.tar只需要 k8s-master-01 节点执行 初始化Master节点
sh
kubeadm init \
--apiserver-advertise-address=192.168.126.170 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.28.0 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--cri-socket=unix:///var/run/cri-dockerd.sock初始化信息
sh
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config只需要 k8s-node-01 、k8s-node-02 节点执行
--cri-socket=unix:///var/run/cri-dockerd.sock 这一句要补充在最后面
shkubeadm join 192.168.111.170:6443 --token y8hujn.777t21thlk6v6hy0 --discovery-token-ca-cert-hash sha256:f8df8dfe6cb7ad5347f92b6c58f552df8982c7dce540b266c22f971e49f55684 --cri-socket=unix:///var/run/cri-dockerd.sock
shkubeadm join 192.168.126.170:6443 --token puuo90.pqgbmu1d32x3vsq4 --discovery-token-ca-cert-hash sha256:3600ff21c5a3742fee0455d920f345ea0ab9c99f0356acf7675f2ee728448bff --cri-socket=unix:///var/run/cri-dockerd.sock
使用kubectl工具查看节点状态： kubectl get nodes 由于网络插件还没有部署，节点会处于"NotReady"状态
这里使用Calico作为Kubernetes的网络插件，负责集群中网络通信。创建Calico网络组件的资源
sh
kubectl create -f tigera-operator.yaml
kubectl create -f custom-resources.yaml 执行完成后，等待几分钟，执行
sh
kubectl get pods -n calico-system -o wide问题排查，发现 nodes 无法启动，原因是下载不了镜像
shkubectl describe po calico-kube-controllers-85955d4f5b-dbhhr -n calico-system kubectl describe po calico-node-2sdxr -n calico-system kubectl describe po calico-node-65gw4 -n calico-system kubectl describe po calico-node-xqvnf -n calico-system kubectl describe po calico-typha-7cd7bb8d58-lqmxj -n calico-system kubectl describe po calico-typha-7cd7bb8d58-zwjd4 -n calico-system kubectl describe po csi-node-driver-d9vkx -n calico-system kubectl describe po csi-node-driver-nl26v -n calico-system kubectl describe po csi-node-driver-qzljb -n calico-system
shsystemctl daemon-reload && systemctl restart docker
shdocker pull registry.cn-beijing.aliyuncs.com/yuncenliu/cni:v3.25.1-calico docker tag registry.cn-beijing.aliyuncs.com/yuncenliu/cni:v3.25.1-calico docker.io/calico/pod2daemon-flexvol:v3.25.1 docker pull registry.cn-beijing.aliyuncs.com/yuncenliu/pod2daemon-flexvol:v3.25.1-calico docker tag registry.cn-beijing.aliyuncs.com/yuncenliu/pod2daemon-flexvol:v3.25.1-calico docker.io/calico/cni:v3.25.1重启containerd （有用）
undefinedsystemctl restart containerd删除失败的 pods
shkubectl get pods -n kube-system | grep calico-node-bvvhc | awk '{print$1}'| xargs kubectl delete -n kube-system pods
shdocker save -o calico.v3.25.1.tar calico/kube-controllers:v3.25.1 calico/node:v3.25.1 calico/pod2daemon-flexvol:v3.25.1 calico/cni:v3.25.1 docker load -i calico.v3.25.1.tar
全部启动成功，节点也全都在线 kubectl get nodes
安装页面
所有节点先下拉镜像
sh
docker load -i k8s-dashboard-v2.7.0.tar创建 Dashboard ，在 master 节点执行
kubectl apply -f kubernetes-dashboard.yaml
kubectl get pods -n kubernetes-dashboard -o wide
执行完成后，任意 https://任意node:30001 访问，例如：https://192.168.111.170:30001/#/login
创建用户
sh
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard用户授权
sh
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin获取用户 token
sh
kubectl create token dashboard-admin -n kubernetes-dashboard
将 token 粘贴到浏览器的 token输入栏中（默认还是选第一项）点击登录即可
