云原生Kubernetes: K8S 1.29版本 部署Harbor

cronaldo912024-05-06 15:28

目录

一、实验

1.环境

[2.Linux 部署docker compose](#2.Linux 部署docker compose)

3.证书秘钥配置

[4.K8S 1.29版本 部署Harbor](#4.K8S 1.29版本 部署Harbor)

[5.K8S 1.29版本 使用Harbor](#5.K8S 1.29版本 使用Harbor)

二、问题

[1.docker 登录harbor失败](#1.docker 登录harbor失败)

一、实验

1.环境

（1）主机

表1 主机

|--------|--------------|--------|----------------|------------|
| 主机 | 架构 | 版本 | IP | 备注 |
| master | K8S master节点 | 1.29.0 | 192.168.204.8 | |
| node1 | K8S node节点 | 1.29.0 | 192.168.204.9 | |
| node2 | K8S node节点 | 1.29.0 | 192.168.204.10 | 已部署Kuboard |

（2）master节点查看集群

bash 复制代码 
1）查看node
kubectl get node
 
2)查看node详细信息
kubectl get node -o wide

（3）查看pod

bash 复制代码 
[root@master ~]# kubectl get pod -A

(4) 访问Kuboard

bash 复制代码 
http://192.168.204.10:30080/kuboard/cluster

查看节点

（5）查看docker版本

bash 复制代码 
docker -v

master节点

node1节点

node2节点

2.Linux 部署docker compose

(1) 查阅

bash 复制代码 
https://github.com/docker/compose/releases

最新版为v2.27.0

(2) 下载

bash 复制代码 
https://github.com/docker/compose/releases/download/v2.27.0/docker-compose-linux-x86_64

(3) 查看

bash 复制代码 
[root@master ~]# ls | grep docker

（4）移动并更名为docker-compose

bash 复制代码 
[root@master ~]# mv docker-compose-linux-x86_64 /usr/bin/docker-compose

（5）为二进制文件添加可执行权限

bash 复制代码 
[root@master ~]# chmod +x /usr/bin/docker-compose

（6）查看docker-compse版本

bash 复制代码 
[root@master ~]# docker-compose version

3.证书秘钥配置

（1）确认openssl

bash 复制代码 
[root@master harbor]# rpm -q openssl

(2)创建文件夹用于存放生成的证书秘钥文件

bash 复制代码 
[root@master ~]# mkdir -p /harbor/certs/

(3) 生成私钥

bash 复制代码 
[root@master ~]# openssl genrsa -out ca.key 4096

（4）证书秘钥与IP绑定

bash 复制代码 
openssl req -x509 -new -nodes -sha512 -days 3650 \
 -subj "/C=CN/ST=Nanjing/L=Nanjing/O=example/OU=Personal/CN=192.168.204.8" \
 -key ca.key \
 -out ca.crt

（5）移动证书位置并查看

bash 复制代码 
[root@master ~]# mv ca* /harbor/certs
[root@master ~]# ls /harbor/certs/

4.K8S 1.29版本 部署Harbor

(1) 查阅

bash 复制代码 
https://github.com/goharbor/harbor/releases

最新版为v2.10.2

(2) 下载

bash 复制代码 
https://github.com/goharbor/harbor/releases/download/v2.10.2/harbor-offline-installer-v2.10.2.tgz

（3）查看

bash 复制代码 
[root@master ~]# ls | grep harbor

（4）解压

bash 复制代码 
[root@master ~]# tar xf harbor-offline-installer-v2.10.2.tgz

（6）进入目录并查看

bash 复制代码 
[root@master ~]# cd harbor/
[root@master harbor]# ls

（7） 修改名称

bash 复制代码 
[root@master harbor]# mv harbor.yml.tmpl harbor.yml

（8）监听端口

8800与8443没有被占用

bash 复制代码 
[root@master harbor]# netstat -antlp | grep 8800
[root@master harbor]# netstat -antlp | grep 8443

（9）修改配置文件

bash 复制代码 
[root@master harbor]# vim harbor.yml
bash 复制代码 
hostname域名修改为要安装harbor的服务器的ip
http端口修改为8800
https端口修改为8443
certificate: 证书路径修改为/harbor/certs/ca.crt
private_key: 秘钥路径修改为/harbor/certs/ca.key
harbor_admin_password: 默认Harbor密码，不修改，就用默认的Harbor12345
data_value: 修改为/data/harbor

修改前：

修改后：

（10）Docker Hub查看镜像

bash 复制代码 
docker pull goharbor/prepare:v2.10.2

(11)拉取镜像

node2

bash 复制代码 
[root@node2 ~]# docker pull goharbor/prepare:v2.10.2

(12) 导出镜像

bash 复制代码 
[root@node2 ~]# docker save -o prepare.tar goharbor/prepare:v2.10.2

(13）复制Docker镜像到master节点

bash 复制代码 
[root@node2 ~]# scp prepare.tar root@master:~

(14）master节点导入Docker镜像

bash 复制代码 
[root@master ~]# docker load -i prepare.tar

（15）执行预备脚本

bash 复制代码 
[root@master harbor]# ./prepare

(16) 执行安装脚本

bash 复制代码 
[root@master harbor]# ./install.sh 

[Step 0]: checking if docker is installed ...

Note: docker version: 24.0.7

[Step 1]: checking docker-compose is installed ...

Note: Docker Compose version v2.21.0

[Step 2]: loading Harbor images ...
a8840e4ae316: Loading layer [==================================================>]  21.63MB/21.63MB
515f0ee642d6: Loading layer [==================================================>]  173.8MB/173.8MB
7e1a15fa7f7c: Loading layer [==================================================>]   25.5MB/25.5MB
4cd39a5aa67d: Loading layer [==================================================>]  18.27MB/18.27MB
3daae6a42996: Loading layer [==================================================>]   5.12kB/5.12kB
e23a6fbc3acf: Loading layer [==================================================>]  6.144kB/6.144kB
87a67eef40a6: Loading layer [==================================================>]  3.072kB/3.072kB
5439373f0cf5: Loading layer [==================================================>]  2.048kB/2.048kB
44c50cb14a57: Loading layer [==================================================>]   2.56kB/2.56kB
e343431b8ac0: Loading layer [==================================================>]   7.68kB/7.68kB
Loaded image: goharbor/harbor-db:v2.10.2
a7437080fdff: Loading layer [==================================================>]  17.16MB/17.16MB
89c54c70338a: Loading layer [==================================================>]  3.584kB/3.584kB
927d190fe457: Loading layer [==================================================>]   2.56kB/2.56kB
dab52f006605: Loading layer [==================================================>]  44.92MB/44.92MB
abc8dcfeeb98: Loading layer [==================================================>]  45.71MB/45.71MB
Loaded image: goharbor/harbor-jobservice:v2.10.2
9343d1011e8e: Loading layer [==================================================>]  21.63MB/21.63MB
d8aa785c45d8: Loading layer [==================================================>]  110.6MB/110.6MB
af30438adf19: Loading layer [==================================================>]  3.072kB/3.072kB
046baf104eef: Loading layer [==================================================>]   59.9kB/59.9kB
b4bf035573f1: Loading layer [==================================================>]  61.95kB/61.95kB
Loaded image: goharbor/redis-photon:v2.10.2
1a53aa673223: Loading layer [==================================================>]  14.15MB/14.15MB
e3ef818740f3: Loading layer [==================================================>]  4.096kB/4.096kB
f51244ac7c03: Loading layer [==================================================>]  3.072kB/3.072kB
971632584e63: Loading layer [==================================================>]   17.4MB/17.4MB
0cfb29c07835: Loading layer [==================================================>]  18.19MB/18.19MB
Loaded image: goharbor/registry-photon:v2.10.2
9fb8de59ccbd: Loading layer [==================================================>]  130.9MB/130.9MB
b0a99212add7: Loading layer [==================================================>]  3.584kB/3.584kB
c63a00cfdba3: Loading layer [==================================================>]  3.072kB/3.072kB
ea1dbece5b3a: Loading layer [==================================================>]   2.56kB/2.56kB
3c735fb3805e: Loading layer [==================================================>]  3.072kB/3.072kB
1b257c0b8709: Loading layer [==================================================>]  3.584kB/3.584kB
6bb174e7db78: Loading layer [==================================================>]  20.48kB/20.48kB
Loaded image: goharbor/harbor-log:v2.10.2
ea9d3f5fbb0c: Loading layer [==================================================>]  121.2MB/121.2MB
b88bdbd268ea: Loading layer [==================================================>]  6.532MB/6.532MB
a6a0154f827a: Loading layer [==================================================>]  246.8kB/246.8kB
664d5baa31ef: Loading layer [==================================================>]  1.477MB/1.477MB
Loaded image: goharbor/harbor-portal:v2.10.2
fe6dfc816265: Loading layer [==================================================>]  17.16MB/17.16MB
c8d1d3bbdf04: Loading layer [==================================================>]  3.584kB/3.584kB
a95b761bfdbe: Loading layer [==================================================>]   2.56kB/2.56kB
c3c092020a00: Loading layer [==================================================>]  58.96MB/58.96MB
75c8aba5c8aa: Loading layer [==================================================>]  5.632kB/5.632kB
84bdcacb7f37: Loading layer [==================================================>]  123.4kB/123.4kB
c80d23d5cead: Loading layer [==================================================>]  80.38kB/80.38kB
fb9a83422043: Loading layer [==================================================>]  59.95MB/59.95MB
17416c18f4d3: Loading layer [==================================================>]   2.56kB/2.56kB
Loaded image: goharbor/harbor-core:v2.10.2
6a65c62b44a3: Loading layer [==================================================>]  14.15MB/14.15MB
50d4a017d046: Loading layer [==================================================>]  4.096kB/4.096kB
b00285e03979: Loading layer [==================================================>]   17.4MB/17.4MB
bd20c4f45bc4: Loading layer [==================================================>]  3.072kB/3.072kB
cdef8cae2c49: Loading layer [==================================================>]  32.77MB/32.77MB
8c8e4274f70f: Loading layer [==================================================>]  50.95MB/50.95MB
Loaded image: goharbor/harbor-registryctl:v2.10.2
acbecb34a764: Loading layer [==================================================>]  17.16MB/17.16MB
478163dbc994: Loading layer [==================================================>]  27.56MB/27.56MB
4ec3465523a2: Loading layer [==================================================>]  4.608kB/4.608kB
07fb78bc4c52: Loading layer [==================================================>]  28.35MB/28.35MB
Loaded image: goharbor/harbor-exporter:v2.10.2
113051e862a0: Loading layer [==================================================>]  121.2MB/121.2MB
Loaded image: goharbor/nginx-photon:v2.10.2
2716f555462f: Loading layer [==================================================>]  14.67MB/14.67MB
d4471d1781eb: Loading layer [==================================================>]  4.096kB/4.096kB
2b9cfa166f1f: Loading layer [==================================================>]  3.072kB/3.072kB
4943bbb50494: Loading layer [==================================================>]  209.9MB/209.9MB
1f70ad7459fb: Loading layer [==================================================>]  17.45MB/17.45MB
c6de541f3b98: Loading layer [==================================================>]  228.1MB/228.1MB
Loaded image: goharbor/trivy-adapter-photon:v2.10.2
Loaded image: goharbor/prepare:v2.10.2


[Step 3]: preparing environment ...

[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir


Note: stopping existing Harbor instance ...


[Step 5]: starting Harbor ...
[+] Running 10/10
 ✔ Network harbor_harbor        Created                                                                                                                                      0.1s 
 ✔ Container harbor-log         Started                                                                                                                                      0.0s 
 ✔ Container registry           Started                                                                                                                                      0.1s 
 ✔ Container registryctl        Started                                                                                                                                      0.1s 
 ✔ Container harbor-portal      Started                                                                                                                                      0.1s 
 ✔ Container harbor-db          Started                                                                                                                                      0.1s 
 ✔ Container redis              Started                                                                                                                                      0.1s 
 ✔ Container harbor-core        Started                                                                                                                                      0.0s 
 ✔ Container nginx              Started                                                                                                                                      0.0s 
 ✔ Container harbor-jobservice  Started                                                                                                                                      0.0s 
✔ ----Harbor has been installed and started successfully.----

(17) 查看harbor相关容器

bash 复制代码 
[root@master harbor]# docker ps | grep  goharbor

（18）docker-compose命令显示其管理的harbor容器

bash 复制代码 
[root@master harbor]# docker-compose -f docker-compose.yml ps

（19）访问harbor

bash 复制代码 
https://192.168.204.8:8443/

选高级

继续访问

弹出界面

（20）登录

bash 复制代码 
账号：admin
密码：Harbor12345

成功进入系统

5. K8S 1.29版本 使用Harbor

(1) 创建私有项目

(2) 查看

目前项目为空

(3) 创建公开项目

（4）再次查看

目前项目为空

（5）Docker配置Harbor（所有节点）

bash 复制代码 
1）修改
vim /etc/docker/daemon.json
......
 "insecure-registries": ["https://192.168.204.8:8443"]
......

2）重新加载配置
systemctl daemon-reload

3）重启服务
systemctl restart docker

systemctl restart cri-docker

修改前：

修改后：

这里以node1节点为例

bash 复制代码 
[root@node1 ~]# systemctl status cri-docker

（6）docker 登录harbor并推送镜像到公开项目

node1节点

bash 复制代码 
[root@node1 ~]# docker login 192.168.204.8:8443 -u admin -p Harbor12345

拉取镜像

打标签

bash 复制代码 
[root@node1 ~]# docker tag alpine:latest 192.168.204.8:8443/devops02/alpine:20240502

推送镜像到公开项目

bash 复制代码 
[root@node1 ~]# docker push 192.168.204.8:8443/devops02/alpine:20240502

查看

删除本地镜像

bash 复制代码 
[root@node1 ~]# docker rmi -f 192.168.204.8:8443/devops02/alpine:20240502

拉取harbor仓库的镜像

bash 复制代码 
[root@node1 ~]# docker pull 192.168.204.8:8443/devops02/alpine:20240502

（7）docker 登录harbor并推送镜像到私有项目

打标签

bash 复制代码 
[root@node1 ~]# docker tag alpine:latest 192.168.204.8:8443/devops01/alpine:20240502

推送镜像到私有项目

bash 复制代码 
[root@node1 ~]# docker push 192.168.204.8:8443/devops01/alpine:20240502

查看

删除本地镜像

bash 复制代码 
[root@node1 ~]# docker rmi -f 192.168.204.8:8443/devops01/alpine:20240502

拉取harbor仓库的镜像

bash 复制代码 
[root@node1 ~]# docker pull 192.168.204.8:8443/devops01/alpine:20240502

(8) 登出账号

bash 复制代码 
[root@node1 ~]# docker logout 192.168.204.8

（9）其他方式的harbor部署

可以参考本人博客：

Docker容器与虚拟化技术：Harbor私有仓库部署与迁移-CSDN博客

云原生Kubernetes：K8S集群使用带凭证的harbor仓库-CSDN博客

二、问题

1.docker 登录harbor失败

（1）报错

bash 复制代码 
Error response from daemon: Get "https://192.168.204.8:8443/v2/": tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead

（2）原因分析

未重启docker。

（3）解决方法

重启docker:

bash 复制代码 
[root@node1 ~]# systemctl restart docker

成功：

相关推荐
weixin_453965001 小时前
[单master节点k8s部署]31.ceph分布式存储（二）
分布式·ceph·kubernetes
漫无目的行走的月亮2 小时前
在Docker中运行微服务注册中心Eureka
docker
大G哥4 小时前
记一次K8S 环境应用nginx stable-alpine 解析内部域名失败排查思路
运维·nginx·云原生·容器·kubernetes
feng_xiaoshi4 小时前
【云原生】云原生架构的反模式
云原生·架构
妍妍的宝贝4 小时前
k8s 中微服务之 MetailLB 搭配 ingress-nginx 实现七层负载
nginx·微服务·kubernetes
大道归简5 小时前
Docker 命令从入门到入门：从 Windows 到容器的完美类比
windows·docker·容器
zeruns8025 小时前
如何搭建自己的域名邮箱服务器？Poste.io邮箱服务器搭建教程，Linux+Docker搭建邮件服务器的教程
linux·运维·服务器·docker·网站
爱跑步的程序员~5 小时前
Docker
docker·容器
福大大架构师每日一题6 小时前
23.1 k8s监控中标签relabel的应用和原理
java·容器·kubernetes
程序那点事儿6 小时前
k8s 之动态创建pv失败（踩坑）
云原生·容器·kubernetes
热门推荐
01分享几个惊艳的 Three.js 个人站点02【经验分享】Ubuntu22.04安装微信(linux官方版)032024年高教社杯数学建模国赛C题超详细解题思路分析04组基轨迹建模 GBTM的介绍与实现（Stata 或 R）05安卓系列机型永久去除data分区加密 详细步骤解析06【2024数模国赛赛题思路公开】国赛B题思路丨附可运行代码丨无偿自提07springer 在线投稿编译踩坑08Ubuntu24.04安装中文输入法09【2024高教社杯全国大学生数学建模竞赛】B题 生产过程中的决策问题——解题思路 代码 论文10CTF网络安全大赛简单的web抓包题目：HEADache