Kubernetes介绍，官网：Kubernetes
- 应用部署方式演变
- 传统部署：互联网早期，会直接将应用程序部署在物理机上
优点：简单，不需要其他技术的参与
缺点：不能为应用程序定义资源使用边界，很难合理地分配计算资源，而且程序之间容器产生影响
- 虚拟化部署：可以在一台物理机上运行多个虚拟机，每个虚拟机都是独立的一个环境
优点：程序环境不会相互产生影响，提供了一定程度的安全性
缺点： 增加了操作系统，浪费了部分资源
- 容器化部署：与虚拟化类似，但是共享了操作系统
优点：可以保证每个容器拥有自己的文件系统、CPU、内存、进程空间等。
运行应用程序所需要的资源都被容器包装，并和底层基础架构解耦。
容器化的应用程序可以跨云服务商、跨Linux操作系统发行版进行部署。
容器化部署方式给带来很多便利，但是也会出现一些问题，比如说：
- 一个容器故障停机了，怎么样让另外一个容器立刻启动去替补停机的容器
- 当并发访问量变大的时候，怎么样做到横向扩展容器数量
这些容器管理的问题统称为容器编排问题，为了解决这些容器编排问题，就产生了一些容器编排的软件：
Swarm：docker自己的容器编排工具
Mesos：apache的一个资源统一管控的工具，需要和marathon结合使用
Kubernetes：google开源的容器编排工具
- kubernetes简介
Kubernetes，是一个全新的基于容器技术的分布式架构领先方案，是谷歌严格保密十几年的秘密武器---Borg系统的一个开源版本，于2014年9月发布第一个版本，2015年7月发布第二个正式版本。
Kubernetes的本质是一组服务器集群，它可以在集群的每个节点上运行特定的程序，来对节点中的容器进行管理。目的是实现资源管理的自动化，主要提供了如下的主要功能：
- 自我修复：一旦某个容器崩溃，能够在1秒中左右迅速启动新的容器
- 弹性伸缩：可以根据需要，自动对集群中正在运行的容器数量进行调整。
- 服务发现：服务可以通过自动发现的形式找到它所依赖的服务
- 负载均衡：如果一个服务启动了多个容器，能够自动实现请求的负载均衡
- 版本回退：如果发现新发布的程序版本问题，可以立即回退到原来的版本
- 存储编排：可以根据容器自身的需求自动创建存储卷
三、kubernetes组件
一个kubernetes集群主要是由控制节点（master）、工作节点（node）构成，每个节点上都会安装不同的组件。
Master：集群的控制平面，负责集群的决策（管理）
Apiserver：资源操作的唯一入口，接收用户输入的命令，提供认证、授权、API注册和发现等机制。
Scheduler：负责集群资源调度，按照预定的调度策略将pod调度到相应的node节点上。
ControllerManager：负责维护集群的状态，比如程序部署安排、故障检测、自动扩展、滚动更新等。
Etcd：负责存储集群中各种资源对象的信息。
Node：集群的数据平面，负责为容器提供运行环境（干活）
Kubelet：负责维护容器的生命周期，即通过控制docker来创建、更新、销毁容器
Docker：负责节点上容器的各种操作
下面以部署一个nginx服务来说明kubernetes系统各个组件的调用关系：
- 首先要明确，一旦kubernetes环境启动之后，master和node都会想自身的信息存储导etcd数据库中
- 一个nginx服务的安装请求会首先被发送到master节点的apiserver组件
- Apiserver组件会调用scheduler组件来决定到底应该把这个服务安装导哪个node节点上
- Apiserver调用controller-manager去调度node节点安装nginx服务
- Kubelet接收到指令后，会通知docker，然后由docker来启动一个nginx的pod，pod是kubernetes的最小操作单元，容器必须跑在pod中
一个nginx服务就运行了，如果需要访问nginx，就需要通过kube-proxy来对pod产生访问的代理，这样外界用户就可以访问集群中的nginx服务了。
四、kubernetes概念
Master：集群控制节点，每个集群需要至少一个master节点负责集群的管控
Node：工作负载节点，由master分配容器到这些node工作节点上，然后node节点上的docker负责容器的运行
Pod：kubernetes的最小控制单元，容器都是运行在pod中的，一个pod中可以有1个或者多个容器
Controller：控制器，通过它来实现对pod的管理，比如启动pod、停止pod、伸缩pod的数量等等
Service：pod对外服务的统一入口，下面可以维护者同一类的多个pod
Label：标签，用于对pod进行分类，同一类pod会拥有相同的标签
Namespace：命名空间，用来隔离pod的运行环境
k8s集群环境搭建
三台主机
192.168.35.100 master
192.168.35.10 node1
192.168.35.20 node2
cpp
//禁用firewalld、selinux、postfix（三个节点都做）
[root@master ~]# systemctl status firewalld.service
○ firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; p>
Active: inactive (dead)
Docs: man:firewalld(1)
[root@master ~]# getenforce
Disabled
//时间同步（三个节点都做）
[root@master ~]# systemctl restart chronyd
[root@master ~]# hwclock -w
[root@master ~]# timedatectl
Local time: Mon 2024-09-23 14:34:16 CST
Universal time: Mon 2024-09-23 06:34:16 UTC
RTC time: Mon 2024-09-23 06:34:16
Time zone: Asia/Shanghai (CST, +0800)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
//
[root@master ~]# vim /etc/hosts
[root@master ~]# scp /etc/hosts root@node1:/etc/hosts
The authenticity of host 'node1 (192.168.35.10)' can't be established.
ED25519 key fingerprint is SHA256:Dfp5NFaBDNajythr8/O8kI1H85ulzHbjA7dnXB4AHfg.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'node1' (ED25519) to the list of known hosts.
root@node1's password:
hosts 100% 220 264.6KB/s 00:00
[root@master ~]# scp /etc/hosts root@node2:/etc/hosts
The authenticity of host 'node2 (192.168.35.20)' can't be established.
ED25519 key fingerprint is SHA256:Dfp5NFaBDNajythr8/O8kI1H85ulzHbjA7dnXB4AHfg.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:1: node1
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'node2' (ED25519) to the list of known hosts.
root@node2's password:
hosts 100% 220 250.1KB/s 00:00
//
[root@master ~]# ssh-keygen
//
[root@master ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@node1
//
[root@master ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@node2
//禁用swap分区（三个节点都做）
[root@master ~]# vim /etc/fstab
[root@master ~]# swapoff -a
[root@master ~]# free -m
total used free shared buff/cache available
Mem: 3883 542 3362 9 202 3341
Swap: 0 0 0
//开启IP转发，和修改内核信息---三个节点都需要配置
[root@master ~]# vim /etc/sysctl.d/k8s.conf
[root@master ~]# modprobe br_netfilter
[root@master ~]# lsmod | grep br_netfilter
br_netfilter 36864 0
bridge 409600 1 br_netfilter
[root@master ~]# sysctl -p /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
//配置IPVS功能（三个节点都做）
[root@master ~]# vim /etc/modules-load.d/ipvs.modules
[root@master ~]# chmod +x /etc/modules-load.d/ipvs.modules
[root@master ~]# bash /etc/modules-load.d/ipvs.modules
[root@master ~]# lsmod | grep ip_vs
ip_vs_sh 16384 0
ip_vs_wrr 16384 0
ip_vs_rr 16384 0
ip_vs 237568 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack 217088 1 ip_vs
nf_defrag_ipv6 24576 2 nf_conntrack,ip_vs
libcrc32c 16384 3 nf_conntrack,xfs,ip_vs
//安装docker
[root@master ~]# yum install -y yum-utils device-mapper-persistent-data lvm2Last metadata
//配置镜像加速器(三台都需要做)
[root@master ~]# vim /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerpull.com",
"https://docker.anyhub.us.kg",
"https://dockerhub.jobcher.com",
"https://dockerhub.icu",
"https://docker.awsl9527.cn"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart docker
[root@master ~]# systemctl enable docker
// 下载cri-docker
[root@master ~]# rz -E
rz waiting to receive.
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd-0.3.9.amd64.tgz
[root@master ~]# tar -zxvf cri-dockerd-0.3.9.amd64.tgz -C /usr/local/bin/
cri-dockerd/
cri-dockerd/cri-dockerd
//下载cri-docker service文件
[root@master bin]# wget -O /etc/systemd/system/cri-docker.service
[root@master bin]# wget -O /etc/systemd/system/cri-docker.socket
//编辑cri-docker.server
[root@master bin]# cd /etc/systemd/system/
[root@master system]# ll | grep cri-docker
-rw-r--r-- 1 root root 1319 Sep 23 15:31 cri-docker.service
-rw-r--r-- 1 root root 204 Sep 23 15:31 cri-docker.socket
[root@master system]# vim cri-docker.service
修改ExecStart行内容为
ExecStart=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9 --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --cri-dockerd-root-directory=/var/lib/dockershim --cri-dockerd-root-directory=/var/lib/docker
//编辑cri-docker.socket
[root@master system]# vim cri-docker.socket
修改ListenStream行内容为
ListenStream=/var/run/cri-dockerd.sock
//复制cri-dockerd到其他节点
[root@master system]# scp cri-docker.s* root@node1:/etc/systemd/system/
cri-docker.service 100% 1524 2.0MB/s 00:00
cri-docker.socket 100% 210 413.1KB/s 00:00
[root@master system]# scp cri-docker.s* root@node2:/etc/systemd/system/
cri-docker.service 100% 1524 1.1MB/s 00:00
cri-docker.socket 100% 210 553.1KB/s 00:00
//设置开机自启
[root@master system]# systemctl daemon-reload
[root@master system]# systemctl restart cri-docker
[root@master system]# systemctl enable cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /etc/systemd/system/cri-docker.service.
//部署k8s源(三台都需要做)
[root@master system]# vim /etc/yum.repos.d/k8s.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/repodata/repomd.xml.key
//安装集群所需软件包kubelet kubeadm kubectl（全部节点）
[root@master system]# yum install -y kubelet kubeadm kubectl
//配置k8s Cgoup控制组（全部节点）
[root@master ~]# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
[root@master ~]# systemctl restart kubelet.service
[root@master ~]# systemctl enable kubelet.service
//配置kubelet自启动（全部节点）
[root@master ~]# kubeadm config images list
//打印集群初始化配置文件（master节点运行
[root@master ~]# kubeadm config print init-defaults > kubeadm-config.yaml
//修改参数（master节点）
[root@master ~]# vim kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.100.10 #修改，集群初始化的主节点IP
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/cri-dockerd.sock #修改使用docke
imagePullPolicy: IfNotPresent
name: master #修改节点名称
taints: null
//使用配置文件初始化（master节点运行）
[root@master ~]# yum -y install socat
[root@master ~]# kubeadm init --config kubeadm-config.yaml --upload-certs
//配置环境变量（master节点运行）
[root@master ~]# vim /etc/profile.d/k8s.sh
export KUBECONFIG=/etc/kubernetes/admin.conf
[root@master ~]# source /etc/profile.d/k8s.sh
//传给另外两台主机
[root@master ~]# scp /etc/profile.d/k8s.sh root@node1:/etc/profile.d/
k8s.sh 100% 45 39.2KB/s 00:00
[root@master ~]# scp /etc/profile.d/k8s.sh root@node2:/etc/profile.d/
k8s.sh 100% 45 81.8KB/s 00:00
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 8m10s v1.28.14
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 20m v1.28.14
node1.example.com NotReady <none> 38s v1.28.14
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 20m v1.28.14
node1.example.com NotReady <none> 83s v1.28.14
node2 NotReady <none> 8s v1.28.14
[root@master ~]#
[root@master ~]# scp /etc/kubernetes/admin.conf root@node1:/etc/kubernetes/
admin.conf 100% 5646 3.6MB/s 00:00
[root@master ~]# scp /etc/kubernetes/admin.conf root@node2:/etc/kubernetes/
admin.conf 100% 5646 3.6MB/s 00:00
//下载calico文件（master节点运行）
[root@master ~]# rz -E
rz waiting to receive.
[root@master ~]# ls
anaconda-ks.cfg calico.yaml cri-dockerd-0.3.9.amd64.tgz kubeadm-config.yaml
[root@master ~]# vim calico.yaml
//创建calico网络（master节点运行）
[root@master ~]# kubectl apply -f calico.yaml
[root@master ~]# kubectl get pods -n kubeadm-system[root@master ~]# kubectl create deployment nginx --image nginx
deployment.apps/nginx created
[root@master ~]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 0/1 1 0 17s
[root@master ~]# kubectl expose deployment nginx --port 80 --type NodePort
service/nginx exposed
[root@master ~]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 90s
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-7854ff8877-6hzd5 1/1 Running 0 106s
[root@master ~]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 77m
nginx NodePort 10.100.174.174 <none> 80:30394/TCP 43s
[root@master ~]#
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-7854ff8877-6hzd5 1/1 Running 0 2m47s 172.16.104.2 node2 <none> <none>
[root@master ~]# curl http://172.16.104.2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@master ~]# kubectl exec -it pod/nginx-7854ff8877-6hzd5 -- /bin/bash
root@nginx-7854ff8877-6hzd5:/# cd /usr/share/nginx/html/
root@nginx-7854ff8877-6hzd5:/usr/share/nginx/html# ls
50x.html index.html
root@nginx-7854ff8877-6hzd5:/usr/share/nginx/html# echo lsy > index.html
root@nginx-7854ff8877-6hzd5:/usr/share/nginx/html# exit
exit
[root@master ~]# curl http://172.16.104.2
lsy
[root@master ~]#
cpp
shell 7 (Build 0164)
Copyright (c) 2020 NetSarang Computer, Inc. All rights reserved.
Type `help' to learn how to use Xshell prompt.
[C:\~]$
Connecting to 192.168.35.10:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.
WARNING! The remote SSH server rejected X11 forwarding request.
Last login: Mon Sep 23 14:39:01 2024 from 192.168.35.100
[root@node1 ~]# bash /etc/modules-load.d/ipvs.modules
[root@node1 ~]#
[root@node1 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
Extra Packages for Enterprise Linux 9 - x86 8.9 kB/s | 10 kB 00:01
Extra Packages for Enterprise Linux 9 - x86 13 MB/s | 23 MB 00:01
Extra Packages for Enterprise Linux 9 openh 2.3 kB/s | 993 B 00:00
Rocky Linux 9 - BaseOS 3.9 kB/s | 4.1 kB 00:01
Rocky Linux 9 - BaseOS 1.6 MB/s | 2.3 MB 00:01
Rocky Linux 9 - AppStream 4.3 kB/s | 4.5 kB 00:01
Rocky Linux 9 - AppStream 5.1 MB/s | 8.0 MB 00:01
Rocky Linux 9 - CRB 2.2 kB/s | 4.5 kB 00:02
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repoRocky Linux 9 - CR [ === ] --- B/s | 0Rocky Linux 9 - CRB 1.7 MB/s | 2.4 MB 00:01
Rocky Linux 9 - Extras 2.8 kB/s | 2.9 kB 00:01
Rocky Linux 9 - Extras 11 kB/s | 15 kB 00:01
Dependencies resolved.
============================================================================
Package Arch Version Repo Size
============================================================================
Installing:
device-mapper-persistent-data x86_64 1.0.9-3.el9_4 baseos 1.0 M
lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M
yum-utils noarch 4.3.0-13.el9 baseos 35 k
Installing dependencies:
device-mapper-event x86_64 9:1.02.197-2.el9 baseos 33 k
device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 31 k
libaio x86_64 0.3.111-13.el9 baseos 23 k
lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M
Transaction Summary
============================================================================
Install 7 Packages
Total download size: 3.6 M
Installed size: 9.8 M
Downloading Packages:
(1/7): libaio-0.3.111-13.el9.x86_64.rpm 190 kB/s | 23 kB 00:00
(2/7): yum-utils-4.3.0-13.el9.noarch.rpm 263 kB/s | 35 kB 00:00
(3/7): lvm2-libs-2.03.23-2.el9.x86_64.rpm 4.1 MB/s | 1.0 MB 00:00
(4/7): device-mapper-persistent-data-1.0.9- 2.6 MB/s | 1.0 MB 00:00
(5/7): device-mapper-event-libs-1.02.197-2. 871 kB/s | 31 kB 00:00
(6/7): lvm2-2.03.23-2.el9.x86_64.rpm 5.5 MB/s | 1.5 MB 00:00
(7/7): device-mapper-event-1.02.197-2.el9.x 702 kB/s | 33 kB 00:00
----------------------------------------------------------------------------
Total 2.6 MB/s | 3.6 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 1/7
Installing : libaio-0.3.111-13.el9.x86_64 2/7
Installing : device-mapper-event-9:1.02.197-2.el9.x86_64 3/7
Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 3/7
Created symlink /etc/systemd/system/sockets.target.wants/dm-event.socket → /usr/lib/systemd/system/dm-event.socket.
Installing : lvm2-libs-9:2.03.23-2.el9.x86_64 4/7
Installing : device-mapper-persistent-data-1.0.9-3.el9_4.x86_ 5/7
Installing : lvm2-9:2.03.23-2.el9.x86_64 6/7
Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 6/7
Created symlink /etc/systemd/system/sysinit.target.wants/lvm2-monitor.service → /usr/lib/systemd/system/lvm2-monitor.service.
Created symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmpolld.socket → /usr/lib/systemd/system/lvm2-lvmpolld.socket.
Installing : yum-utils-4.3.0-13.el9.noarch 7/7
Running scriptlet: yum-utils-4.3.0-13.el9.noarch 7/7
Verifying : libaio-0.3.111-13.el9.x86_64 1/7
Verifying : yum-utils-4.3.0-13.el9.noarch 2/7
Verifying : device-mapper-persistent-data-1.0.9-3.el9_4.x86_ 3/7
Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 4/7
Verifying : lvm2-9:2.03.23-2.el9.x86_64 5/7
Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 6/7
Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 7/7
Installed:
device-mapper-event-9:1.02.197-2.el9.x86_64
device-mapper-event-libs-9:1.02.197-2.el9.x86_64
device-mapper-persistent-data-1.0.9-3.el9_4.x86_64
libaio-0.3.111-13.el9.x86_64
lvm2-9:2.03.23-2.el9.x86_64
lvm2-libs-9:2.03.23-2.el9.x86_64
yum-utils-4.3.0-13.el9.noarch
Complete!
[root@node1 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Adding repo from: https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@node1 ~]# sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
[root@node1 ~]# yum -y install docker-ce-25.0.5-1.el9
Waiting for process with pid 10702 to finish.
Last metadata expiration check: 0:00:01 ago on Mon 23 Sep 2024 03:12:43 PM CST.
Dependencies resolved.
============================================================================
Package Arch Version Repository Size
============================================================================
Installing:
docker-ce x86_64 3:25.0.5-1.el9 docker-ce-stable 26 M
Installing dependencies:
container-selinux noarch 3:2.229.0-1.el9 appstream 56 k
containerd.io x86_64 1.7.22-3.1.el9 docker-ce-stable 43 M
docker-ce-cli x86_64 1:27.3.1-1.el9 docker-ce-stable 8.0 M
fuse-common x86_64 3.10.2-8.el9 baseos 7.3 k
fuse-overlayfs x86_64 1.13-1.el9 appstream 66 k
fuse3 x86_64 3.10.2-8.el9 appstream 53 k
fuse3-libs x86_64 3.10.2-8.el9 appstream 90 k
libslirp x86_64 4.4.0-7.el9 appstream 68 k
slirp4netns x86_64 1.2.3-1.el9 appstream 46 k
tar x86_64 2:1.34-6.el9_4.1 baseos 876 k
Installing weak dependencies:
docker-buildx-plugin x86_64 0.17.1-1.el9 docker-ce-stable 14 M
docker-ce-rootless-extras x86_64 27.3.1-1.el9 docker-ce-stable 4.4 M
docker-compose-plugin x86_64 2.29.7-1.el9 docker-ce-stable 13 M
Transaction Summary
============================================================================
Install 14 Packages
Total download size: 109 M
Installed size: 424 M
Downloading Packages:
(1/14): docker-buildx-plugin-0.17.1-1.el9.x 11 MB/s | 14 MB 00:01
(2/14): docker-ce-25.0.5-1.el9.x86_64.rpm 12 MB/s | 26 MB 00:02
(3/14): docker-ce-cli-27.3.1-1.el9.x86_64.r 7.4 MB/s | 8.0 MB 00:01
(4/14): docker-ce-rootless-extras-27.3.1-1. 6.4 MB/s | 4.4 MB 00:00
(5/14): tar-1.34-6.el9_4.1.x86_64.rpm 2.1 MB/s | 876 kB 00:00
(6/14): fuse-common-3.10.2-8.el9.x86_64.rpm 185 kB/s | 7.3 kB 00:00
(7/14): container-selinux-2.229.0-1.el9.noa 1.3 MB/s | 56 kB 00:00
(8/14): fuse-overlayfs-1.13-1.el9.x86_64.rp 1.3 MB/s | 66 kB 00:00
(9/14): slirp4netns-1.2.3-1.el9.x86_64.rpm 851 kB/s | 46 kB 00:00
(10/14): libslirp-4.4.0-7.el9.x86_64.rpm 1.5 MB/s | 68 kB 00:00
(11/14): fuse3-libs-3.10.2-8.el9.x86_64.rpm 1.8 MB/s | 90 kB 00:00
(12/14): fuse3-3.10.2-8.el9.x86_64.rpm 1.3 MB/s | 53 kB 00:00
(13/14): docker-compose-plugin-2.29.7-1.el9 7.6 MB/s | 13 MB 00:01
(14/14): containerd.io-1.7.22-3.1.el9.x86_6 10 MB/s | 43 MB 00:04
----------------------------------------------------------------------------
Total 18 MB/s | 109 MB 00:06
Docker CE Stable - x86_64 13 kB/s | 1.6 kB 00:00
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
From : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : fuse3-libs-3.10.2-8.el9.x86_64 1/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 2/14
Installing : container-selinux-3:2.229.0-1.el9.noarch 2/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 2/14
Installing : docker-compose-plugin-2.29.7-1.el9.x86_64 3/14
Running scriptlet: docker-compose-plugin-2.29.7-1.el9.x86_64 3/14
Installing : containerd.io-1.7.22-3.1.el9.x86_64 4/14
Running scriptlet: containerd.io-1.7.22-3.1.el9.x86_64 4/14
Installing : libslirp-4.4.0-7.el9.x86_64 5/14
Installing : slirp4netns-1.2.3-1.el9.x86_64 6/14
Installing : fuse-common-3.10.2-8.el9.x86_64 7/14
Installing : fuse3-3.10.2-8.el9.x86_64 8/14
Installing : fuse-overlayfs-1.13-1.el9.x86_64 9/14
Running scriptlet: fuse-overlayfs-1.13-1.el9.x86_64 9/14
Installing : tar-2:1.34-6.el9_4.1.x86_64 10/14
Installing : docker-buildx-plugin-0.17.1-1.el9.x86_64 11/14
Running scriptlet: docker-buildx-plugin-0.17.1-1.el9.x86_64 11/14
Installing : docker-ce-cli-1:27.3.1-1.el9.x86_64 12/14
Running scriptlet: docker-ce-cli-1:27.3.1-1.el9.x86_64 12/14
Installing : docker-ce-rootless-extras-27.3.1-1.el9.x86_64 13/14
Running scriptlet: docker-ce-rootless-extras-27.3.1-1.el9.x86_64 13/14
Installing : docker-ce-3:25.0.5-1.el9.x86_64 14/14
Running scriptlet: docker-ce-3:25.0.5-1.el9.x86_64 14/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 14/14
Running scriptlet: docker-ce-3:25.0.5-1.el9.x86_64 14/14
Verifying : containerd.io-1.7.22-3.1.el9.x86_64 1/14
Verifying : docker-buildx-plugin-0.17.1-1.el9.x86_64 2/14
Verifying : docker-ce-3:25.0.5-1.el9.x86_64 3/14
Verifying : docker-ce-cli-1:27.3.1-1.el9.x86_64 4/14
Verifying : docker-ce-rootless-extras-27.3.1-1.el9.x86_64 5/14
Verifying : docker-compose-plugin-2.29.7-1.el9.x86_64 6/14
Verifying : tar-2:1.34-6.el9_4.1.x86_64 7/14
Verifying : fuse-common-3.10.2-8.el9.x86_64 8/14
Verifying : container-selinux-3:2.229.0-1.el9.noarch 9/14
Verifying : fuse-overlayfs-1.13-1.el9.x86_64 10/14
Verifying : slirp4netns-1.2.3-1.el9.x86_64 11/14
Verifying : libslirp-4.4.0-7.el9.x86_64 12/14
Verifying : fuse3-libs-3.10.2-8.el9.x86_64 13/14
Verifying : fuse3-3.10.2-8.el9.x86_64 14/14
Installed:
container-selinux-3:2.229.0-1.el9.noarch
containerd.io-1.7.22-3.1.el9.x86_64
docker-buildx-plugin-0.17.1-1.el9.x86_64
docker-ce-3:25.0.5-1.el9.x86_64
docker-ce-cli-1:27.3.1-1.el9.x86_64
docker-ce-rootless-extras-27.3.1-1.el9.x86_64
docker-compose-plugin-2.29.7-1.el9.x86_64
fuse-common-3.10.2-8.el9.x86_64
fuse-overlayfs-1.13-1.el9.x86_64
fuse3-3.10.2-8.el9.x86_64
fuse3-libs-3.10.2-8.el9.x86_64
libslirp-4.4.0-7.el9.x86_64
slirp4netns-1.2.3-1.el9.x86_64
tar-2:1.34-6.el9_4.1.x86_64
Complete!
[root@node1 ~]# vim /etc/docker/daemon.json
[root@node1 ~]#
[root@node1 ~]# systemctl daemon-reload
[root@node1 ~]# systemctl restart docker
[root@node1 ~]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
[root@node1 ~]# ls
anaconda-ks.cfg cri-dockerd-0.3.9.amd64.tgz
[root@node1 ~]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
cri-dockerd/cri-dockerd
[root@node1 ~]# ll /etc/systemd/system
total 16
drwxr-xr-x. 2 root root 31 Jul 11 09:53 basic.target.wants
-rw-r--r-- 1 root root 1524 Sep 23 16:38 cri-docker.service
-rw-r--r-- 1 root root 210 Sep 23 16:38 cri-docker.socket
lrwxrwxrwx. 1 root root 37 Jul 11 09:52 ctrl-alt-del.target -> /usr/lib/systemd/system/reboot.target
lrwxrwxrwx. 1 root root 57 Jul 11 09:52 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service
lrwxrwxrwx. 1 root root 43 Jul 11 09:52 dbus.service -> /usr/lib/systemd/system/dbus-broker.service
lrwxrwxrwx. 1 root root 41 Jul 11 09:54 default.target -> /usr/lib/systemd/system/multi-user.target
drwxr-xr-x. 2 root root 32 Jul 11 09:52 getty.target.wants
drwxr-xr-x. 2 root root 4096 Sep 23 15:18 multi-user.target.wants
drwxr-xr-x. 2 root root 48 Jul 11 09:52 network-online.target.wants
drwxr-xr-x. 2 root root 71 Sep 23 15:10 sockets.target.wants
drwxr-xr-x. 2 root root 4096 Sep 23 15:10 sysinit.target.wants
drwxr-xr-x. 2 root root 56 Jul 11 09:53 timers.target.wants
[root@node1 ~]# systemctl daemon-reload
[root@node1 ~]# systemctl restart cri-docker
[root@node1 ~]# systemctl enable cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /etc/systemd/system/cri-docker.service.
[root@node1 ~]# vim /etc/yum.repos.d/k8s.repo
[root@node1 ~]# yum install -y kubelet kubeadm kubectl
Kubernetes 139 kB/s | 35 kB 00:00
Dependencies resolved.
============================================================================
Package Arch Version Repository Size
============================================================================
Installing:
kubeadm x86_64 1.28.14-150500.2.1 kubernetes 9.8 M
kubectl x86_64 1.28.14-150500.2.1 kubernetes 10 M
kubelet x86_64 1.28.14-150500.2.1 kubernetes 19 M
Installing dependencies:
conntrack-tools x86_64 1.4.7-2.el9 appstream 221 k
cri-tools x86_64 1.28.0-150500.1.1 kubernetes 8.1 M
kubernetes-cni x86_64 1.2.0-150500.2.1 kubernetes 6.2 M
libnetfilter_cthelper x86_64 1.0.0-22.el9 appstream 23 k
libnetfilter_cttimeout x86_64 1.0.0-19.el9 appstream 23 k
libnetfilter_queue x86_64 1.0.5-1.el9 appstream 28 k
Transaction Summary
============================================================================
Install 9 Packages
Total download size: 53 M
Installed size: 290 M
Downloading Packages:
(1/9): kubeadm-1.28.14-150500.2.1.x86_64.rp 9.6 MB/s | 9.8 MB 00:01
(2/9): cri-tools-1.28.0-150500.1.1.x86_64.r 7.8 MB/s | 8.1 MB 00:01
(3/9): kubectl-1.28.14-150500.2.1.x86_64.rp 9.0 MB/s | 10 MB 00:01
(4/9): conntrack-tools-1.4.7-2.el9.x86_64.r 793 kB/s | 221 kB 00:00
(5/9): libnetfilter_cttimeout-1.0.0-19.el9. 434 kB/s | 23 kB 00:00
(6/9): libnetfilter_cthelper-1.0.0-22.el9.x 279 kB/s | 23 kB 00:00
(7/9): libnetfilter_queue-1.0.5-1.el9.x86_6 808 kB/s | 28 kB 00:00
(8/9): kubernetes-cni-1.2.0-150500.2.1.x86_ 8.4 MB/s | 6.2 MB 00:00
(9/9): kubelet-1.28.14-150500.2.1.x86_64.rp 9.9 MB/s | 19 MB 00:01
----------------------------------------------------------------------------
Total 11 MB/s | 53 MB 00:04
Kubernetes 11 kB/s | 1.7 kB 00:00
Importing GPG key 0x9A296436:
Userid : "isv:kubernetes OBS Project <isv:kubernetes@build.opensuse.org>"
Fingerprint: DE15 B144 86CD 377B 9E87 6E1A 2346 54DA 9A29 6436
From : https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/repodata/repomd.xml.key
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : kubernetes-cni-1.2.0-150500.2.1.x86_64 1/9
Installing : libnetfilter_queue-1.0.5-1.el9.x86_64 2/9
Installing : libnetfilter_cthelper-1.0.0-22.el9.x86_64 3/9
Installing : libnetfilter_cttimeout-1.0.0-19.el9.x86_64 4/9
Installing : conntrack-tools-1.4.7-2.el9.x86_64 5/9
Running scriptlet: conntrack-tools-1.4.7-2.el9.x86_64 5/9
Installing : kubelet-1.28.14-150500.2.1.x86_64 6/9
Running scriptlet: kubelet-1.28.14-150500.2.1.x86_64 6/9
Installing : kubectl-1.28.14-150500.2.1.x86_64 7/9
Installing : cri-tools-1.28.0-150500.1.1.x86_64 8/9
Installing : kubeadm-1.28.14-150500.2.1.x86_64 9/9
Running scriptlet: kubeadm-1.28.14-150500.2.1.x86_64 9/9
Verifying : cri-tools-1.28.0-150500.1.1.x86_64 1/9
Verifying : kubeadm-1.28.14-150500.2.1.x86_64 2/9
Verifying : kubectl-1.28.14-150500.2.1.x86_64 3/9
Verifying : kubelet-1.28.14-150500.2.1.x86_64 4/9
Verifying : kubernetes-cni-1.2.0-150500.2.1.x86_64 5/9
Verifying : conntrack-tools-1.4.7-2.el9.x86_64 6/9
Verifying : libnetfilter_cttimeout-1.0.0-19.el9.x86_64 7/9
Verifying : libnetfilter_cthelper-1.0.0-22.el9.x86_64 8/9
Verifying : libnetfilter_queue-1.0.5-1.el9.x86_64 9/9
Installed:
conntrack-tools-1.4.7-2.el9.x86_64
cri-tools-1.28.0-150500.1.1.x86_64
kubeadm-1.28.14-150500.2.1.x86_64
kubectl-1.28.14-150500.2.1.x86_64
kubelet-1.28.14-150500.2.1.x86_64
kubernetes-cni-1.2.0-150500.2.1.x86_64
libnetfilter_cthelper-1.0.0-22.el9.x86_64
libnetfilter_cttimeout-1.0.0-19.el9.x86_64
libnetfilter_queue-1.0.5-1.el9.x86_64
Complete!
[root@node1 ~]# vim /etc/sysconfig/kubelet
[root@node1 ~]# systemctl restart kubelet.service
[root@node1 ~]# systemctl enable kubelet.service
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.
[root@node1 ~]#
[root@node1 ~]# source /etc/profile.d/k8s.sh
[root@node1 ~]# kubeadm join 192.168.35.100:6443 --token abcdef.0123456789abcdef \
[root@node1 ~]# kubeadm join 192.168.35.100:6443 --token abcdef.0123456789abcdef \
> --discovery-token-ca-cert-hash sha256:c2323748d44cd1c6193fa6f9c15deb56566696fdc66f2bb2c25a52c4f968babb --cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[WARNING FileExisting-socat]: socat not found in system path
[WARNING Hostname]: hostname "node1.example.com" could not be reached
[WARNING Hostname]: hostname "node1.example.com": lookup node1.example.com on 114.114.114.114:53: no such host
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@node1 ~]# kubectl get nodes
E0923 17:23:47.547511 14341 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E0923 17:23:47.548048 14341 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E0923 17:23:47.549911 14341 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E0923 17:23:47.550538 14341 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E0923 17:23:47.552162 14341 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@node1 ~]# kubeadm join 192.168.35.100:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:c2323748d44cd1c6193fa6f9c15deb56566696fdc66f2bb2c25a52c4f968babb --cri-socket=unix:///var/run/cri-dockerd.sock
[root@node1 ~]# cd /etc/kubernetes/
[root@node1 kubernetes]# ls
kubelet.conf manifests pki
[root@node1 kubernetes]# source /etc/profile.d/k8s.sh
[root@node1 kubernetes]# cd
[root@node1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 25m v1.28.14
node1.example.com NotReady <none> 6m17s v1.28.14
node2 NotReady <none> 5m2s v1.28.14
[root@node1 ~]#
cpp
Xshell 7 (Build 0164)
Copyright (c) 2020 NetSarang Computer, Inc. All rights reserved.
Type `help' to learn how to use Xshell prompt.
[C:\~]$
Connecting to 192.168.35.20:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.
WARNING! The remote SSH server rejected X11 forwarding request.
Last login: Mon Sep 23 14:39:10 2024 from 192.168.35.100
[root@node2 ~]# bash /etc/modules-load.d/ipvs.modules
[root@node2 ~]#
[root@node2 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
Last metadata expiration check: 0:20:55 ago on Mon 23 Sep 2024 02:48:45 PM CST.
Dependencies resolved.
============================================================================
Package Arch Version Repo Size
============================================================================
Installing:
device-mapper-persistent-data x86_64 1.0.9-3.el9_4 baseos 1.0 M
lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M
yum-utils noarch 4.3.0-13.el9 baseos 35 k
Installing dependencies:
device-mapper-event x86_64 9:1.02.197-2.el9 baseos 33 k
device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 31 k
libaio x86_64 0.3.111-13.el9 baseos 23 k
lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M
Transaction Summary
============================================================================
Install 7 Packages
Total download size: 3.6 M
Installed size: 9.8 M
Downloading Packages:
(1/7): yum-utils-4.3.0-13.el9.noarch.rpm 284 kB/s | 35 kB 00:00
(2/7): libaio-0.3.111-13.el9.x86_64.rpm 171 kB/s | 23 kB 00:00
(3/7): device-mapper-persistent-data-1.0.9- 2.6 MB/s | 1.0 MB 00:00
(4/7): device-mapper-event-libs-1.02.197-2. 842 kB/s | 31 kB 00:00
(5/7): lvm2-libs-2.03.23-2.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00
(6/7): lvm2-2.03.23-2.el9.x86_64.rpm 5.1 MB/s | 1.5 MB 00:00
(7/7): device-mapper-event-1.02.197-2.el9.x 1.0 MB/s | 33 kB 00:00
----------------------------------------------------------------------------
Total 2.5 MB/s | 3.6 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 1/7
Installing : libaio-0.3.111-13.el9.x86_64 2/7
Installing : device-mapper-event-9:1.02.197-2.el9.x86_64 3/7
Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 3/7
Created symlink /etc/systemd/system/sockets.target.wants/dm-event.socket → /usr/lib/systemd/system/dm-event.socket.
Installing : lvm2-libs-9:2.03.23-2.el9.x86_64 4/7
Installing : device-mapper-persistent-data-1.0.9-3.el9_4.x86_ 5/7
Installing : lvm2-9:2.03.23-2.el9.x86_64 6/7
Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 6/7
Created symlink /etc/systemd/system/sysinit.target.wants/lvm2-monitor.service → /usr/lib/systemd/system/lvm2-monitor.service.
Created symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmpolld.socket → /usr/lib/systemd/system/lvm2-lvmpolld.socket.
Installing : yum-utils-4.3.0-13.el9.noarch 7/7
Running scriptlet: yum-utils-4.3.0-13.el9.noarch 7/7
Verifying : libaio-0.3.111-13.el9.x86_64 1/7
Verifying : yum-utils-4.3.0-13.el9.noarch 2/7
Verifying : device-mapper-persistent-data-1.0.9-3.el9_4.x86_ 3/7
Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 4/7
Verifying : lvm2-9:2.03.23-2.el9.x86_64 5/7
Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 6/7
Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 7/7
Installed:
device-mapper-event-9:1.02.197-2.el9.x86_64
device-mapper-event-libs-9:1.02.197-2.el9.x86_64
device-mapper-persistent-data-1.0.9-3.el9_4.x86_64
libaio-0.3.111-13.el9.x86_64
lvm2-9:2.03.23-2.el9.x86_64
lvm2-libs-9:2.03.23-2.el9.x86_64
yum-utils-4.3.0-13.el9.noarch
Complete!
[root@node2 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Adding repo from: https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@node2 ~]# sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
[root@node2 ~]# yum -y install docker-ce-25.0.5-1.el9
Docker CE Stable - x86_64 136 kB/s | 58 kB 00:00
Dependencies resolved.
============================================================================
Package Arch Version Repository Size
============================================================================
Installing:
docker-ce x86_64 3:25.0.5-1.el9 docker-ce-stable 26 M
Installing dependencies:
container-selinux noarch 3:2.229.0-1.el9 appstream 56 k
containerd.io x86_64 1.7.22-3.1.el9 docker-ce-stable 43 M
docker-ce-cli x86_64 1:27.3.1-1.el9 docker-ce-stable 8.0 M
fuse-common x86_64 3.10.2-8.el9 baseos 7.3 k
fuse-overlayfs x86_64 1.13-1.el9 appstream 66 k
fuse3 x86_64 3.10.2-8.el9 appstream 53 k
fuse3-libs x86_64 3.10.2-8.el9 appstream 90 k
libslirp x86_64 4.4.0-7.el9 appstream 68 k
slirp4netns x86_64 1.2.3-1.el9 appstream 46 k
tar x86_64 2:1.34-6.el9_4.1 baseos 876 k
Installing weak dependencies:
docker-buildx-plugin x86_64 0.17.1-1.el9 docker-ce-stable 14 M
docker-ce-rootless-extras x86_64 27.3.1-1.el9 docker-ce-stable 4.4 M
docker-compose-plugin x86_64 2.29.7-1.el9 docker-ce-stable 13 M
Transaction Summary
============================================================================
Install 14 Packages
Total download size: 109 M
Installed size: 424 M
Downloading Packages:
(1/14): docker-buildx-plugin-0.17.1-1.el9.x 10 MB/s | 14 MB 00:01
(2/14): docker-ce-25.0.5-1.el9.x86_64.rpm 10 MB/s | 26 MB 00:02
(3/14): docker-ce-cli-27.3.1-1.el9.x86_64.r 5.6 MB/s | 8.0 MB 00:01
(4/14): docker-ce-rootless-extras-27.3.1-1. 5.8 MB/s | 4.4 MB 00:00
(5/14): tar-1.34-6.el9_4.1.x86_64.rpm 2.1 MB/s | 876 kB 00:00
(6/14): fuse-common-3.10.2-8.el9.x86_64.rpm 243 kB/s | 7.3 kB 00:00
(7/14): container-selinux-2.229.0-1.el9.noa 2.0 MB/s | 56 kB 00:00
(8/14): fuse-overlayfs-1.13-1.el9.x86_64.rp 1.2 MB/s | 66 kB 00:00
(9/14): slirp4netns-1.2.3-1.el9.x86_64.rpm 1.0 MB/s | 46 kB 00:00
(10/14): libslirp-4.4.0-7.el9.x86_64.rpm 1.8 MB/s | 68 kB 00:00
(11/14): fuse3-libs-3.10.2-8.el9.x86_64.rpm 2.3 MB/s | 90 kB 00:00
(12/14): fuse3-3.10.2-8.el9.x86_64.rpm 1.5 MB/s | 53 kB 00:00
(13/14): docker-compose-plugin-2.29.7-1.el9 9.3 MB/s | 13 MB 00:01
(14/14): containerd.io-1.7.22-3.1.el9.x86_6 9.3 MB/s | 43 MB 00:04
----------------------------------------------------------------------------
Total 17 MB/s | 109 MB 00:06
Docker CE Stable - x86_64 12 kB/s | 1.6 kB 00:00
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
From : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : fuse3-libs-3.10.2-8.el9.x86_64 1/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 2/14
Installing : container-selinux-3:2.229.0-1.el9.noarch 2/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 2/14
Installing : docker-compose-plugin-2.29.7-1.el9.x86_64 3/14
Running scriptlet: docker-compose-plugin-2.29.7-1.el9.x86_64 3/14
Installing : containerd.io-1.7.22-3.1.el9.x86_64 4/14
Running scriptlet: containerd.io-1.7.22-3.1.el9.x86_64 4/14
Installing : libslirp-4.4.0-7.el9.x86_64 5/14
Installing : slirp4netns-1.2.3-1.el9.x86_64 6/14
Installing : fuse-common-3.10.2-8.el9.x86_64 7/14
Installing : fuse3-3.10.2-8.el9.x86_64 8/14
Installing : fuse-overlayfs-1.13-1.el9.x86_64 9/14
Running scriptlet: fuse-overlayfs-1.13-1.el9.x86_64 9/14
Installing : tar-2:1.34-6.el9_4.1.x86_64 10/14
Installing : docker-buildx-plugin-0.17.1-1.el9.x86_64 11/14
Running scriptlet: docker-buildx-plugin-0.17.1-1.el9.x86_64 11/14
Installing : docker-ce-cli-1:27.3.1-1.el9.x86_64 12/14
Running scriptlet: docker-ce-cli-1:27.3.1-1.el9.x86_64 12/14
Installing : docker-ce-rootless-extras-27.3.1-1.el9.x86_64 13/14
Running scriptlet: docker-ce-rootless-extras-27.3.1-1.el9.x86_64 13/14
Installing : docker-ce-3:25.0.5-1.el9.x86_64 14/14
Running scriptlet: docker-ce-3:25.0.5-1.el9.x86_64 14/14
Running scriptlet: container-selinux-3:2.229.0-1.el9.noarch 14/14
Running scriptlet: docker-ce-3:25.0.5-1.el9.x86_64 14/14
Verifying : containerd.io-1.7.22-3.1.el9.x86_64 1/14
Verifying : docker-buildx-plugin-0.17.1-1.el9.x86_64 2/14
Verifying : docker-ce-3:25.0.5-1.el9.x86_64 3/14
Verifying : docker-ce-cli-1:27.3.1-1.el9.x86_64 4/14
Verifying : docker-ce-rootless-extras-27.3.1-1.el9.x86_64 5/14
Verifying : docker-compose-plugin-2.29.7-1.el9.x86_64 6/14
Verifying : tar-2:1.34-6.el9_4.1.x86_64 7/14
Verifying : fuse-common-3.10.2-8.el9.x86_64 8/14
Verifying : container-selinux-3:2.229.0-1.el9.noarch 9/14
Verifying : fuse-overlayfs-1.13-1.el9.x86_64 10/14
Verifying : slirp4netns-1.2.3-1.el9.x86_64 11/14
Verifying : libslirp-4.4.0-7.el9.x86_64 12/14
Verifying : fuse3-libs-3.10.2-8.el9.x86_64 13/14
Verifying : fuse3-3.10.2-8.el9.x86_64 14/14
Installed:
container-selinux-3:2.229.0-1.el9.noarch
containerd.io-1.7.22-3.1.el9.x86_64
docker-buildx-plugin-0.17.1-1.el9.x86_64
docker-ce-3:25.0.5-1.el9.x86_64
docker-ce-cli-1:27.3.1-1.el9.x86_64
docker-ce-rootless-extras-27.3.1-1.el9.x86_64
docker-compose-plugin-2.29.7-1.el9.x86_64
fuse-common-3.10.2-8.el9.x86_64
fuse-overlayfs-1.13-1.el9.x86_64
fuse3-3.10.2-8.el9.x86_64
fuse3-libs-3.10.2-8.el9.x86_64
libslirp-4.4.0-7.el9.x86_64
slirp4netns-1.2.3-1.el9.x86_64
tar-2:1.34-6.el9_4.1.x86_64
Complete!
[root@node2 ~]# vim /etc/docker/daemon.json
[root@node2 ~]#
[root@node2 ~]# systemctl daemon-reload
[root@node2 ~]# systemctl restart docker
[root@node2 ~]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
[root@node2 ~]# ls
anaconda-ks.cfg cri-dockerd-0.3.9.amd64.tgz
[root@node2 ~]# cd /usr/local/bin/
[root@node2 bin]# ls
[root@node2 bin]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
tar: cri-dockerd-0.3.9.amd64.tgz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
[root@node2 bin]# systemctl daemon-reload
[root@node2 bin]# systemctl restart cri-docker
Job for cri-docker.service failed because the control process exited with error code.
See "systemctl status cri-docker.service" and "journalctl -xeu cri-docker.service" for details.
[root@node2 bin]# ll /etc/systemd/system
total 16
drwxr-xr-x. 2 root root 31 Jul 11 09:53 basic.target.wants
-rw-r--r-- 1 root root 1524 Sep 23 16:38 cri-docker.service
-rw-r--r-- 1 root root 210 Sep 23 16:38 cri-docker.socket
lrwxrwxrwx. 1 root root 37 Jul 11 09:52 ctrl-alt-del.target -> /usr/lib/systemd/system/reboot.target
lrwxrwxrwx. 1 root root 57 Jul 11 09:52 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service
lrwxrwxrwx. 1 root root 43 Jul 11 09:52 dbus.service -> /usr/lib/systemd/system/dbus-broker.service
lrwxrwxrwx. 1 root root 41 Jul 11 09:54 default.target -> /usr/lib/systemd/system/multi-user.target
drwxr-xr-x. 2 root root 32 Jul 11 09:52 getty.target.wants
drwxr-xr-x. 2 root root 4096 Sep 23 15:18 multi-user.target.wants
drwxr-xr-x. 2 root root 48 Jul 11 09:52 network-online.target.wants
drwxr-xr-x. 2 root root 71 Sep 23 15:09 sockets.target.wants
drwxr-xr-x. 2 root root 4096 Sep 23 15:09 sysinit.target.wants
drwxr-xr-x. 2 root root 56 Jul 11 09:53 timers.target.wants
[root@node2 bin]# cd
[root@node2 ~]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
cri-dockerd/cri-dockerd
[root@node2 ~]# systemctl daemon-reload
[root@node2 ~]# systemctl restart cri-docker
[root@node2 ~]# systemctl enable cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /etc/systemd/system/cri-docker.service.
[root@node2 ~]# vim /etc/yum.repos.d/k8s.repo
[root@node2 ~]# yum install -y kubelet kubeadm kubectl
Kubernetes 118 kB/s | 35 kB 00:00
Last metadata expiration check: 0:00:01 ago on Mon 23 Sep 2024 04:46:04 PM CST.
Dependencies resolved.
============================================================================
Package Arch Version Repository Size
============================================================================
Installing:
kubeadm x86_64 1.28.14-150500.2.1 kubernetes 9.8 M
kubectl x86_64 1.28.14-150500.2.1 kubernetes 10 M
kubelet x86_64 1.28.14-150500.2.1 kubernetes 19 M
Installing dependencies:
conntrack-tools x86_64 1.4.7-2.el9 appstream 221 k
cri-tools x86_64 1.28.0-150500.1.1 kubernetes 8.1 M
kubernetes-cni x86_64 1.2.0-150500.2.1 kubernetes 6.2 M
libnetfilter_cthelper x86_64 1.0.0-22.el9 appstream 23 k
libnetfilter_cttimeout x86_64 1.0.0-19.el9 appstream 23 k
libnetfilter_queue x86_64 1.0.5-1.el9 appstream 28 k
Transaction Summary
============================================================================
Install 9 Packages
Total download size: 53 M
Installed size: 290 M
Downloading Packages:
(1/9): kubeadm-1.28.14-150500.2.1.x86_64.rp 6.6 MB/s | 9.8 MB 00:01
(2/9): cri-tools-1.28.0-150500.1.1.x86_64.r 5.1 MB/s | 8.1 MB 00:01
(3/9): kubectl-1.28.14-150500.2.1.x86_64.rp 6.0 MB/s | 10 MB 00:01
(4/9): conntrack-tools-1.4.7-2.el9.x86_64.r 255 kB/s | 221 kB 00:00
(5/9): libnetfilter_cttimeout-1.0.0-19.el9. 1.1 MB/s | 23 kB 00:00
(6/9): libnetfilter_cthelper-1.0.0-22.el9.x 890 kB/s | 23 kB 00:00
(7/9): kubernetes-cni-1.2.0-150500.2.1.x86_ 6.0 MB/s | 6.2 MB 00:01
(8/9): libnetfilter_queue-1.0.5-1.el9.x86_6 1.2 MB/s | 28 kB 00:00
(9/9): kubelet-1.28.14-150500.2.1.x86_64.rp 11 MB/s | 19 MB 00:01
----------------------------------------------------------------------------
Total 12 MB/s | 53 MB 00:04
Kubernetes 13 kB/s | 1.7 kB 00:00
Importing GPG key 0x9A296436:
Userid : "isv:kubernetes OBS Project <isv:kubernetes@build.opensuse.org>"
Fingerprint: DE15 B144 86CD 377B 9E87 6E1A 2346 54DA 9A29 6436
From : https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/repodata/repomd.xml.key
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : kubernetes-cni-1.2.0-150500.2.1.x86_64 1/9
Installing : libnetfilter_queue-1.0.5-1.el9.x86_64 2/9
Installing : libnetfilter_cthelper-1.0.0-22.el9.x86_64 3/9
Installing : libnetfilter_cttimeout-1.0.0-19.el9.x86_64 4/9
Installing : conntrack-tools-1.4.7-2.el9.x86_64 5/9
Running scriptlet: conntrack-tools-1.4.7-2.el9.x86_64 5/9
Installing : kubelet-1.28.14-150500.2.1.x86_64 6/9
Running scriptlet: kubelet-1.28.14-150500.2.1.x86_64 6/9
Installing : kubectl-1.28.14-150500.2.1.x86_64 7/9
Installing : cri-tools-1.28.0-150500.1.1.x86_64 8/9
Installing : kubeadm-1.28.14-150500.2.1.x86_64 9/9
Running scriptlet: kubeadm-1.28.14-150500.2.1.x86_64 9/9
Verifying : cri-tools-1.28.0-150500.1.1.x86_64 1/9
Verifying : kubeadm-1.28.14-150500.2.1.x86_64 2/9
Verifying : kubectl-1.28.14-150500.2.1.x86_64 3/9
Verifying : kubelet-1.28.14-150500.2.1.x86_64 4/9
Verifying : kubernetes-cni-1.2.0-150500.2.1.x86_64 5/9
Verifying : conntrack-tools-1.4.7-2.el9.x86_64 6/9
Verifying : libnetfilter_cttimeout-1.0.0-19.el9.x86_64 7/9
Verifying : libnetfilter_cthelper-1.0.0-22.el9.x86_64 8/9
Verifying : libnetfilter_queue-1.0.5-1.el9.x86_64 9/9
Installed:
conntrack-tools-1.4.7-2.el9.x86_64
cri-tools-1.28.0-150500.1.1.x86_64
kubeadm-1.28.14-150500.2.1.x86_64
kubectl-1.28.14-150500.2.1.x86_64
kubelet-1.28.14-150500.2.1.x86_64
kubernetes-cni-1.2.0-150500.2.1.x86_64
libnetfilter_cthelper-1.0.0-22.el9.x86_64
libnetfilter_cttimeout-1.0.0-19.el9.x86_64
libnetfilter_queue-1.0.5-1.el9.x86_64
Complete!
[root@node2 ~]# vim /etc/sysconfig/kubelet
[root@node2 ~]# systemctl restart kubelet.service
[root@node2 ~]# systemctl enable kubelet.service
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.
[root@node2 ~]#
[root@node2 ~]# source /etc/profile.d/k8s.sh
[root@node2 ~]# kubeadm join 192.168.35.100:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:c2323748d44cd1c6193fa6f9c15deb56566696fdc66f2bb2c25a52c4f968babb --cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[WARNING FileExisting-socat]: socat not found in system path
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@node2 ~]# source /etc/profile.d/k8s.sh
[root@node2 ~]#
[root@node2 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 25m v1.28.14
node1.example.com NotReady <none> 6m28s v1.28.14
node2 NotReady <none> 5m13s v1.28.14
cpp
[root@master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-9d57d8f49-kgvqp 1/1 Running 0 2m33s
calico-node-8blj7 1/1 Running 0 2m33s
calico-node-n4rsr 1/1 Running 0 2m33s
calico-node-qzzhp 1/1 Running 0 2m33s
coredns-6554b8b87f-knzmb 1/1 Running 0 72m
coredns-6554b8b87f-qzzf5 1/1 Running 0 72m
etcd-master 1/1 Running 0 72m
kube-apiserver-master 1/1 Running 0 72m
kube-controller-manager-master 1/1 Running 0 72m
kube-proxy-2sg4f 1/1 Running 0 72m
kube-proxy-94rfz 1/1 Running 0 51m
kube-proxy-p42cj 1/1 Running 0 53m
kube-scheduler-master 1/1 Running 0 72m