K8S Kubernetes集群部署

1、主机规划

复制代码

testk8s-master    192.168.4.10
testk8s-node1    192.168.4.11
testk8s-node2    192.168.4.12

系统配置为4C8G200G，centos7系统，分区为/boot、/，无SWAP分区

2、操作系统初始化-所有节点

关闭防火墙

arduino 复制代码

systemctl stop firewalld
systemctl disable firewalld

关闭selinux

arduino 复制代码

sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0

关闭swap

bash 复制代码

swapoff -a  # 临时
sed -ri 's/.*swap.*/#&/' /etc/fstab    # 永久

在master节点添加hosts

bash 复制代码

cat >> /etc/hosts << EOF
192.168.4.10 testk8s-master
192.168.4.11 testk8s-node1
192.168.4.12 testk8s-node2
EOF

将桥接的IPv4流量传递到iptables的链

bash 复制代码

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system  # 生效

时间同步

bash 复制代码

vi /etc/chrony.conf
增加 server 114.115.116.117 iburst
systemctl restart chronyd
立即同步时间
chronyc -a makestep 
查看同步状态
chronyc tracking

3、安装docker

配置阿里云、清华镜像源

bash 复制代码

curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum install -y yum-utils
sudo yum-config-manager --add-repo https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/centos/docker-ce.repo
sudo sed -i 's|https://download.docker.com|https://mirrors.tuna.tsinghua.edu.cn/docker-ce|g' /etc/yum.repos.d/docker-ce.repo
yum clean all
yum makecache
yum install bash-completion -y
yum install docker-ce -y --nogpgcheck
systemctl enable docker && systemctl start docker
systemctl restart docker
docker info

4、安装vmtools

arduino 复制代码

yum install open-vm-tools -y

5、做快照

防止操作错误

6、安装kubeadm，kubelet和kubectl

配置镜像加速

镜像源列表www.cnblogs.com/gnuorg/p/18...

ini 复制代码

cat > /etc/docker/daemon.json << EOF
{
  "registry-mirrors": ["https://docker.1panel.live"]
}
EOF

 systemctl restart docker

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

指定版本号

bash 复制代码

yum install -y kubelet-1.20.0 kubeadm-1.20.0 kubectl-1.20.0
systemctl enable kubelet

7、部署Kubernetes Master

在192.168.4.10（Master）执行。

css 复制代码

kubeadm init \
  --apiserver-advertise-address=192.168.4.10 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.20.0 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=10.244.0.0/16 \
  --ignore-preflight-errors=all

解释
--apiserver-advertise-address 集群通告地址
--image-repository 由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址
--kubernetes-version K8s版本，与上面安装的一致
--service-cidr 集群内部虚拟网络，Pod统一访问入口
--pod-network-cidr Pod网络，与下面部署的CNI网络组件yaml中保持一致
--ignore-preflight-errors=all 忽略错误
初始化完成后，最后会输出一个join命令，先记住，下面用。

执行后返回

sql 复制代码

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.4.10:6443 --token oweerb.nonsh3zl5a8no0od \
    --discovery-token-ca-cert-hash sha256:279352b82d65dd6bd470ea1b8c54542215696402a0d6bd8a20e53102f39f8a21

拷贝kubectl使用的连接k8s认证文件到默认路径

bash 复制代码

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

查看工作节点

arduino 复制代码

kubectl get nodes
NAME             STATUS     ROLES                  AGE    VERSION
testk8s-master   NotReady   control-plane,master   104s   v1.20.0

8、加入K8S node

在Node节点执行

192.168.4.11

192.168.4.12

向集群添加新节点，执行在kubeadm init输出的kubeadm join命令

sql 复制代码

kubeadm join 192.168.4.10:6443 --token oweerb.nonsh3zl5a8no0od \
    --discovery-token-ca-cert-hash sha256:279352b82d65dd6bd470ea1b8c54542215696402a0d6bd8a20e53102f39f8a21

默认token有效期为24小时，当过期之后，该token就不可用了。这时就需要重新创建token，可以直接使用命令快捷生成

lua 复制代码

kubeadm token create --print-join-command

查看工作节点

sql 复制代码

kubectl get nodes
NAME             STATUS     ROLES                  AGE     VERSION
testk8s-master   NotReady   control-plane,master   3m42s   v1.20.0
testk8s-node1    NotReady   <none>                 19s     v1.20.0
testk8s-node2    NotReady   <none>                 16s     v1.20.0

9、部署容器网络（CNI）

Calico是一个纯三层的数据中心网络方案，是目前Kubernetes主流的网络方案。

下载YAML

bash 复制代码

curl https://docs.projectcalico.org/v3.20/manifests/calico.yaml -O

下载完后还需要修改里面定义Pod网络（CALICO_IPV4POOL_CIDR），与前面kubeadm init的 --pod-network-cidr指定的一样。

yaml 复制代码

# The default IPv4 pool to create on startup if none exists. Pod IPs will be
            # chosen from this range. Changing this value after installation will have
            # no effect. This should fall within `--cluster-cidr`.
            - name: CALICO_IPV4POOL_CIDR
              value: "10.244.0.0/16"

修改完后文件后，部署：

sql 复制代码

kubectl apply -f calico.yaml
kubectl get pods -n kube-system

等Calico Pod都Running，节点也会准备就绪。

注：以后所有yaml文件都只在Master节点执行！

安装目录：/etc/kubernetes/

组件配置文件目录：/etc/kubernetes/manifests/

节点运行情况

sql 复制代码

 kubectl get pods -n kube-system
NAME                                       READY   STATUS     RESTARTS   AGE
calico-kube-controllers-577f77cb5c-jrcfs   0/1     Pending    0          5s
calico-node-fznrr                          0/1     Init:0/3   0          6s
calico-node-nrrwj                          0/1     Init:0/3   0          6s
calico-node-x7hds                          0/1     Init:0/3   0          6s
coredns-7f89b7bc75-6lr2s                   0/1     Pending    0          7m18s
coredns-7f89b7bc75-kwq9c                   0/1     Pending    0          7m18s
etcd-testk8s-master                        1/1     Running    0          7m26s
kube-apiserver-testk8s-master              1/1     Running    0          7m26s
kube-controller-manager-testk8s-master     1/1     Running    0          7m26s
kube-proxy-6pbwh                           1/1     Running    0          4m9s
kube-proxy-btgsz                           1/1     Running    0          4m12s
kube-proxy-cdfxc                           1/1     Running    0          7m18s
kube-scheduler-testk8s-master              1/1     Running    0          7m26s

会出现的一种情况是镜像下载失败

ruby 复制代码

calico-node-fznrr                          0/1     Init:ImagePullBackOff   0          5m32s
calico-node-nrrwj                          0/1     Init:ImagePullBackOff   0          5m32s
calico-node-x7hds                          0/1     Init:ImagePullBackOff   0          5m32s

查看失败原因

vbnet 复制代码

 kubectl describe po calico-node-fznrr -n kube-system
  Warning  Failed     2m11s                 kubelet            Failed to pull image "docker.io/calico/pod2daemon-flexvol:v3.20.6": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed     2m11s                 kubelet            Error: ErrImagePull
  Normal   BackOff    2m11s                 kubelet            Back-off pulling image "docker.io/calico/pod2daemon-flexvol:v3.20.6"
  Warning  Failed     2m11s                 kubelet            Error: ImagePullBackOff
  Normal   Pulling    116s (x2 over 5m31s)  kubelet            Pulling image "docker.io/calico/pod2daemon-flexvol:v3.20.6"

通过镜像站点下载 docker.aityp.com/image/docke...

bash 复制代码

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/calico/pod2daemon-flexvol:v3.20.6
docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/calico/pod2daemon-flexvol:v3.20.6  docker.io/calico/pod2daemon-flexvol:v3.20.6

等待自动修复完成

sql 复制代码

calico-kube-controllers-577f77cb5c-jrcfs   0/1     ContainerCreating   0          22m
calico-node-fznrr                          0/1     Running             0          22m
calico-node-nrrwj                          1/1     Running             0          22m
calico-node-x7hds                          1/1     Running             0          22m

有时发生错误，重启k8s也能解决

复制代码

systemctl restart kubelet

创建pod测试

ini 复制代码

kubectl create deployment nginx --image=nginx

查看pod状态

arduino 复制代码

kubectl get pod

查看pod状态带节点和IP

sql 复制代码

kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP             NODE            NOMINATED NODE   READINESS GATES
nginx-6799fc88d8-rqb82   1/1     Running   0          14m   10.244.236.3   testk8s-node1   <none>           <none>

测试nginx

xml 复制代码

curl 10.244.236.3

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

创建外部访问

css 复制代码

kubectl expose deployment nginx --port=80 --target-port=80 --type=NodePort

查看外部端口范围 30000以上

sql 复制代码

kubectl get pod,svc

NAME                         READY   STATUS    RESTARTS   AGE
pod/nginx-6799fc88d8-rqb82   1/1     Running   0          15m

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        46m
service/nginx        NodePort    10.101.228.228   <none>        80:32507/TCP   15s

访问地址为
http://192.168.4.11:32507/ http://192.168.4.12:32507/

即Pod任意节点IP，组合service映射的端口

10、部署dashboard

YAML下载地址

bash 复制代码

curl https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.3/aio/deploy/recommended.yaml -O

修改yaml，增加nodePort: 30001 type: NodePort

yaml 复制代码

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard

部署dashboard

复制代码

kubectl apply -f recommended.yaml

查看状态

arduino 复制代码

kubectl get pods -n kubernetes-dashboard

创建service account并绑定默认cluster-admin管理员集群角色：

创建用户

sql 复制代码

kubectl create serviceaccount dashboard-admin -n kube-system

用户授权

css 复制代码

kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin

获取用户Token

makefile 复制代码

kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/ {print $1}')

Name:         dashboard-admin-token-sqtsm
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: 77ad4c5d-e4e0-4dc9-b014-7f679acf5aff

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1066 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6InY0U0pqNDh2M0ZGMVdMTGdxSnNBcmxMaVFGVE9nMC1tMnhxQzFfZjF3aEUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tc3F0c20iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzdhZDRjNWQtZTRlMC00ZGM5LWIwMTQtN2Y2NzlhY2Y1YWZmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.fwivtHsitw0ABTfb96HqIJ6N9SL23eiZtIjniqB1qRYIODkGJkOXKGpUmEXPRwR-pQr4glk1KDP9dB2xidET9IhZ-3iKt_5K8xb9K3aELG9yOzzH0Xmi88SaY6A6ZrABaCjjTcp80d-5FgQhRB6ruMLnD1N7vftYk1Sf37HvZ_bKApq1C6uebKnMd0M2EcPckjepvSXmD6fdsosTAJrTYeEpcFCjR6IS5R9bnrN7ADwFZHu-kEekhhV7g888REdhnbSkAvzE9OYbIf7uVgTkh6C_ZhJEzODViHS_RDkiEbZSqs0Q53h50CgL8tj3CBrkV9FvO7SoKVCtvTkYZyPfcQ

访问地址：https://NodeIP:30001

任何节点都可以访问https://192.168.4.10:30001/ https://192.168.4.11:30001/

EDGE访问出现你的连接不是专用链接，没有继续访问按钮时

解决办法

保持焦点在页面内，鼠标在页面空白处点击（不选中任何按钮），直接输入"thisisunsafe"，输完后按回车键，就可以正常访问网页。

这里要注意的是，输入的时候页面时不会有任何反应的，也不会显示输入的字符，是正常现象。输入完毕后点回车即可。

输入Token登录

go 复制代码

token:      
eyJhbGciOiJSUzI1NiIsImtpZCI6InY0U0pqNDh2M0ZGMVdMTGdxSnNBcmxMaVFGVE9nMC1tMnhxQzFfZjF3aEUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tc3F0c20iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzdhZDRjNWQtZTRlMC00ZGM5LWIwMTQtN2Y2NzlhY2Y1YWZmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.fwivtHsitw0ABTfb96HqIJ6N9SL23eiZtIjniqB1qRYIODkGJkOXKGpUmEXPRwR-pQr4glk1KDP9dB2xidET9IhZ-3iKt_5K8xb9K3aELG9yOzzH0Xmi88SaY6A6ZrABaCjjTcp80d-5FgQhRB6ruMLnD1N7vftYk1Sf37HvZ_bKApq1C6uebKnMd0M2EcPckjepvSXmD6fdsosTAJrTYeEpcFCjR6IS5R9bnrN7ADwFZHu-kEekhhV7g888REdhnbSkAvzE9OYbIf7uVgTkh6C_ZhJEzODViHS_RDkiEbZSqs0Q53h50CgL8tj3CBrkV9FvO7SoKVCtvTkYZyPfcQ

11、查看日志

查看容器日志

bash 复制代码

kubectl logs 容器名称 -n kube-system

kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6799fc88d8-rqb82   1/1     Running   0          37m

kubectl logs nginx-6799fc88d8-rqb82
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

查看容器事件

yaml 复制代码

kubectl describe pod 容器名称 -n kube-system

kubectl describe pod nginx-6799fc88d8-rqb82
Name:         nginx-6799fc88d8-rqb82
Namespace:    default
Priority:     0
Node:         testk8s-node1/192.168.4.11
Start Time:   Wed, 22 Oct 2025 14:17:29 +0800
Labels:       app=nginx
              pod-template-hash=6799fc88d8

查看calico.yaml所需要的镜像

arduino 复制代码

 grep image calico.yaml
 
          image: docker.io/calico/cni:v3.20.6
          image: docker.io/calico/cni:v3.20.6
          image: docker.io/calico/pod2daemon-flexvol:v3.20.6
          image: docker.io/calico/node:v3.20.6
          image: docker.io/calico/kube-controllers:v3.20.6

cailco镜像下载失败时解决办法

通过镜像站下载 docker.aityp.com/

清空部署环境

perl 复制代码

kubeadm reset

系统命令补全

复制代码

 yum install bash-completion -y