安装kubeadm、kubectl、和 kubelet

这里通过百度网盘下载所需要的安装包：

链接: k8s部署包.zip_免费高速下载|百度网盘-分享无限制提取码: 0000

1、下载部署包到本地后，在k8s部署包/k8s目录下

执行此yum命令安装：yum localinstall ./*.rpm 或 rpm -Uvh *.rpm --nodeps --force

2、在 /etc/sysconfig/kublet 中增加自定义 kubelet 配置

设置hostname

hostnamectl set-hostname master

mkdir -p /home/work/kubernetes/log

mkdir -p /home/work/kubernetes/kubelet_data

mkdir -p /home/work/docker

自定义kublet配置

vi /etc/sysconfig/kubelet

KUBELET_EXTRA_ARGS=" --hostname-override=master --logtostderr=false --log-dir=/home/work/kubernetes/log --v=0\

--root-dir=/home/work/kubernetes/kubelet_data "

3、设置 kubelet 开机启动

systemctl enable kubelet && systemctl start kubelet

docker安装

1、rpm包在k8s部署包/docker目录下

2、切换到该目录下安装

yum localinstall ./*.rpm 或rpm -Uvh *.rpm --nodeps --force

4、若是GPU机节点，需要安装nvidia-dcoker

1、切到k8s部署包/docker/nvidia-dcoker目录下

3、在该目录下执行

rpm -Uvh *.rpm --nodeps --force

4、sudo systemctl restart docker

5、修改docker配置，包括cgroup和仓库地址

vim /etc/docker/daemon.json

gpu机器:

{

"runtimes": {

"nvidia": {

"path": "nvidia-container-runtime",

"runtimeArgs": []

}

"registry-mirrors": ["https://23h04een.mirror.aliyuncs.com"],

"exec-opts": ["native.cgroupdriver=systemd"],

"default-runtime": "nvidia",

"storage-driver": "overlay2",

"storage-opts": [

"overlay2.override_kernel_check=true"

]

}

cpu机器:

{

"exec-opts": ["native.cgroupdriver=systemd"]

}

6、修改docker服务参数，增加docker配置文件，指定docker数据文件目录

vim /etc/kubernetes/docker_opts.env

DOCKER_OPT_IPMASQ="--ip-masq=true"

DOCKER_OPT_MTU="--mtu=1372"

DOCKER_OPTS=" --ip-masq=true --mtu=1372 --data-root=/home/work/docker "

vim /usr/lib/systemd/system/docker.service (EnvironmentFile 对应上面的文件绝对路径)

EnvironmentFile=/etc/kubernetes/docker_opts.env

ExecStart=/usr/bin/dockerd $DOCKER_OPTS -H fd:// --containerd=/run/containerd/containerd.sock

#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

7、重启docker，设置开机启动

systemctl daemon-reload

systemctl restart docker

nfs准备

安装nfs，配置挂载

先进入到k8s部署包/nfs-utils目录下

执行命令安装：yum localinstall ./*.rpm 或rpm -Uvh *.rpm --nodeps --force

创建本地目录

mkdir /cfs

挂载远程目录到本地

mount -t nfs4 -o minorversion=1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport cfs-PJ5X3SRg7M.lb-d065a956.cfs.bj.baidubce.com:/ /cfs

设置开机启动

systemctl enable nfs.service

更改服务器配置

关闭防火墙和selinux

- configMapRef:

name: config-secret

extraEnvs: []

- name: MY_ENVIRONMENT_VAR

value: the_value_goes_here

extraVolumes:

name: extras

hostPath:

path: /home/work/docker/containers

type: DirectoryOrCreate

extraVolumeMounts:

name: extras

mountPath: /home/work/docker/containers

readOnly: true

hostNetworking: false

such as filebeat.yml for daemonset

filebeatConfig:

filebeat.yml: |

filebeat.inputs:

type: container

paths:

/var/log/containers/*-job-*.log

processors:

add_kubernetes_metadata:

host: ${NODE_NAME}

matchers:

logs_path:

logs_path: "/var/log/containers"

output.elasticsearch:

host: '${NODE_NAME}'

hosts: '${ELASTICSEARCH_HOSTS:10.138.117.28:8200}'

创建namespace

kubectl create ns logs

指定ns安装

helm install filebeat -n logs filebeat/

卸载

helm uninstall filebeat -n logs

注意点：es的地址需要配置对，否则连接不上es

7、job-extension安装

1、需要的yaml文件：job-extension.yaml

tar包：job-extension-vtest2.tar

2、在/k8s部署包/job-extension下

导入镜像：docker load -i job-extension-vtest2.tar

3、kubectl apply -f job-extension.yaml

8、设置任务优先级资源

1、在/k8s部署包/job-extension下

2、kubectl apply -f priorityclass.yaml

生成ca证书脚本

export clientcert=$(grep client-cert ~/.kube/config |cut -d" " -f 6)

export certauth=$(grep certificate-authority-data ~/.kube/config |cut -d" " -f 6)

export clientkey=$(grep client-key-data ~/.kube/config |cut -d" " -f 6)

echo $clientcert | base64 -d > ./client.pem

echo $clientkey | base64 -d > ./client-key.pem

echo $certauth | base64 -d > ./ca.pem

7、部署node节点参考：

k8s 集群新增 Node 文档

8、k8s集群默认存在污点策略NoSchedule，master节点不会被调度到。如果需要master节点被调度，则需要做去除操作：

kubectl taint nodes --all for-node-role.kubernetes.io/master:NoSchedule-

参考：k8s污点策略与容忍的简单应用_prefernoschedule_xhredeem的博客-CSDN博客

k8s最详细集群部署