目录
[3.Jenkins手动测试前端项目CD 流水线代码(下载部署文件)](#3.Jenkins手动测试前端项目CD 流水线代码(下载部署文件))
[4. 将K8S master节点配置为jenkins从节点](#4. 将K8S master节点配置为jenkins从节点)
[5.K8S 手动回滚前端项目版本](#5.K8S 手动回滚前端项目版本)
[6.Jenkins手动测试前端项目CD 流水线代码(发布应用)](#6.Jenkins手动测试前端项目CD 流水线代码(发布应用))
[7.Jenkins手动测试前端项目CD 流水线代码(添加选择回滚功能)](#7.Jenkins手动测试前端项目CD 流水线代码(添加选择回滚功能))
[8. Jenkins创建前端项目CD流水线](#8. Jenkins创建前端项目CD流水线)
[9. GitLab修改代码](#9. GitLab修改代码)
[10. Jenkins手动构建前端项目CI流水线](#10. Jenkins手动构建前端项目CI流水线)
[11. K8S node节点连接Harbor拉取镜像](#11. K8S node节点连接Harbor拉取镜像)
12.Jenkins通过GitLab共享库实现前端项目在K8S自动发布应用
[1. jenkins从节点配置页面显示端口被禁用](#1. jenkins从节点配置页面显示端口被禁用)
[2.将K8S master节点配置为jenkins从节点报错](#2.将K8S master节点配置为jenkins从节点报错)
一、实验
1.环境
(1)主机
表1 主机
|---------|--------------|----------|---------------------|---------------------|
| 主机 | 架构 | 版本 | IP | 备注 |
| master1 | K8S master节点 | 1.20.6 | 192.168.204.180 | jenkins slave (从节点) |
| node1 | K8S node节点 | 1.20.6 | 192.168.204.181 | |
| node2 | K8S node节点 | 1.20.6 | 192.168.204.182 | |
| jenkins | jenkins主节点 | 2.414.2 | 192.168.204.15:8080 | gitlab runner (从节点) |
| | harbor私有仓库 | 1.2.2 | 192.168.204.15 | |
| gitlab | gitlab 主节点 | 12.10.14 | 192.168.204.8:82 | jenkins slave (从节点) |
| | sonarqube | 9.6 | 192.168.204.8:9000 | |
2.GitLab新建项目存放K8S部署文件
(1)新建项目
![](https://file.jishuzhan.net/article/1737941699642003457/5a76c77d01cadee9ffe84f94ce43a406.webp)
(2)K8S查看节点及部署文件
bash
# kubectl get nodes
# cat devops03-devops-ui.yaml
(3)将部署文件复制到GitLab项目
![](https://file.jishuzhan.net/article/1737941699642003457/5280d36d378c557f47d2f47b09eb2bc5.webp)
(4)完成提交
![](https://file.jishuzhan.net/article/1737941699642003457/22e82ea1c267cedea4d7382f253a294d.webp)
(5)查看项目编号为22
![](https://file.jishuzhan.net/article/1737941699642003457/f52e57021d2cd33776cc412e45c75147.webp)
3.Jenkins手动测试前端项目CD 流水线代码(下载部署文件)
(1)在回放中运行测试代码
bash
@Library("mylib@master") _
import org.devops.*
def gitlabbutil = new Gitlab()
pipeline {
agent { label "build"}
options {
skipDefaultCheckout true
}
stages{
stage("GetDeployFile"){
steps{
script {
println("GetCode")
fileData = gitlabbutil.GetRepoFile(22,"deployment.yaml", "master")
println(fileData)
}
}
}
}
}
![](https://file.jishuzhan.net/article/1737941699642003457/7430cb319b460d30c3ea91731cf3bc0e.webp)
(2)成功
![](https://file.jishuzhan.net/article/1737941699642003457/2d5232ecdd96a1060350a69259820b1c.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/a98d1e0603b4529f2d0fe257879c07e3.webp)
(3)生成流水线脚本实现写入文件
![](https://file.jishuzhan.net/article/1737941699642003457/a1b41e0cdd722cb1d4d8f7e0f58ed662.webp)
(4)回放中运行代码测试代码
bash
@Library("mylib@master") _
import org.devops.*
def gitlabbutil = new Gitlab()
pipeline {
agent { label "build"}
options {
skipDefaultCheckout true
}
stages{
stage("GetDeployFile"){
steps{
script {
println("GetCode")
fileData = gitlabbutil.GetRepoFile(22,"deployment.yaml", "master")
println(fileData)
writeFile file: 'deployment.yaml', text: fileData
sh "ls -l; cat deployment.yaml"
}
}
}
}
}
![](https://file.jishuzhan.net/article/1737941699642003457/0e1a626c817b142f4a6cb9e54d9dcb99.webp)
(2)成功
![](https://file.jishuzhan.net/article/1737941699642003457/a52930166746d426eba4c2395bf61fed.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/c45d91db8def1a3515c2c98282327150.webp)
4. 将K8S master节点配置为jenkins从节点
(1)Jenkins添加节点
![](https://file.jishuzhan.net/article/1737941699642003457/f27b9f9892493e123d616a981dad0118.webp)
(2)设置
![](https://file.jishuzhan.net/article/1737941699642003457/6f6ac89b3f880416df4df347fb972d14.webp)
(3)配置界面
![](https://file.jishuzhan.net/article/1737941699642003457/cf58c699d75156c899f5a35419082524.webp)
(4)K8S master节点创建目录下载jar包
bash
# mkdir /tmp/jenkinsagent
# cd /tmp/jenkinsagent
# wget http://192.168.204.15:8080/jnlpJars/agent.jar
![](https://file.jishuzhan.net/article/1737941699642003457/3232a02af7880a7a85b4dbca7a0071f9.webp)
选择第3种方式进行安装
运或者运行脚本
![](https://file.jishuzhan.net/article/1737941699642003457/4924d5700ba13d18e2304bac822cc7e5.webp)
(5)成功添加
![](https://file.jishuzhan.net/article/1737941699642003457/65b03d4a7292414e3641278a3157a8d6.webp)
(6)查看节点列表
K8S 集群关闭了Swap Space
![](https://file.jishuzhan.net/article/1737941699642003457/5ab1e31ed642e8fa49b654c8a839441a.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/cb5bd959ccf96c57d563d394347ce255.webp)
5.K8S 手动回滚前端项目版本
(1)查看历史
# kubectl rollout history deployment/devops03-devops-ui -n devops03
![](https://file.jishuzhan.net/article/1737941699642003457/0e0e92c1cbb76a2981cf85c8bac02cef.webp)
(2)查看具体某一个历史版本信息
bash
# kubectl rollout history deployment/devops03-devops-ui --revision=3 -n devops03
# kubectl rollout history deployment/devops03-devops-ui --revision=4 -n devops03
(3) 另开一个终端用watch命令观察pod变化
# watch -n 1 "kubectl get pod -n devops03"
![](https://file.jishuzhan.net/article/1737941699642003457/e8ff26b2927a6df57457825782f2ae07.webp)
(4)外部测试访问(当前版本为1.1.6)
bash
# curl http://devops03-devops-ui.devops.com:31291
(5)回滚指定版本
bash
# kubectl rollout undo deployment/devops03-devops-ui --to-revision=3 -n devops03
(6)观察pod变化
(7)外部测试访问(回滚版本为1.1.5)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/0e5bf5c9e74aeb95128f19fbe2709776.webp)
6.Jenkins手动测试前端项目CD 流水线代码(发布应用)
(1)回放中运行代码测试代码
bash
@Library("mylib@master") _
import org.devops.*
def gitlabbutil = new Gitlab()
env.groupName = "${JOB_NAME}".split("/")[0]
env.projectName ="${JOB_NAME}".split("/")[-1].split("_")[0]
pipeline {
agent { label "k8s"}
options {
skipDefaultCheckout true
}
stages{
stage("GetDeployFile"){
steps{
script {
println("GetCode")
fileData = gitlabbutil.GetRepoFile(22,"deployment.yaml", "master")
//println(fileData)
sh "rm -fr deployment.yaml"
writeFile file: 'deployment.yaml', text: fileData
//sh "ls -l; cat deployment.yaml"
sh "ls -l "
}
}
}
stage("DeployAPP"){
steps{
script{
namespace = "${env.groupName}"
appName = "${env.projectName}"
sh """
## 发布应用
kubectl apply -f deployment.yaml -n ${namespace}
"""
// 获取应用状态
5.times{
sh "sleep 2; kubectl -n ${namespace} get pod | grep ${appName} "
}
}
}
}
}
}
![](https://file.jishuzhan.net/article/1737941699642003457/60903199d6cfd2273de34e5f7f95a75a.webp)
(2)成功
(3)观察pod变化
(4)外部测试访问(版本升级为1.1.6)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/63493686b07816a17c6e2496236da35a.webp)
7.Jenkins手动测试前端项目CD 流水线代码(添加选择回滚功能)
(1)先手动回滚为1.1.5
bash
# kubectl rollout undo deployment/devops03-devops-ui -n devops03
deployment.apps/devops03-devops-ui rolled back
![](https://file.jishuzhan.net/article/1737941699642003457/0873081c2bc7090f77c78ec74c4d84d7.webp)
(2)观察pod变化
(3) 外部测试访问(当前版本回滚为为1.1.5)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/a7aca6fba25f517b9763aa320d03138b.webp)
(4)回放中运行代码测试代码
bash
@Library("mylib@master") _
import org.devops.*
def gitlabbutil = new Gitlab()
env.groupName = "${JOB_NAME}".split("/")[0]
env.projectName ="${JOB_NAME}".split("/")[-1].split("_")[0]
pipeline {
agent { label "k8s"}
options {
skipDefaultCheckout true
}
stages{
stage("GetDeployFile"){
steps{
script {
println("GetCode")
fileData = gitlabbutil.GetRepoFile(22,"deployment.yaml", "master")
//println(fileData)
sh "rm -fr deployment.yaml"
writeFile file: 'deployment.yaml', text: fileData
//sh "ls -l; cat deployment.yaml"
sh "ls -l "
}
}
}
stage("DeployAPP"){
steps{
script{
env.namespace = "${env.groupName}"
env.appName = "${env.projectName}"
sh """
## 发布应用
kubectl apply -f deployment.yaml -n ${env.namespace}
"""
// 获取应用状态
5.times{
sh "sleep 2; kubectl -n ${env.namespace} get pod | grep ${env.appName} "
}
}
}
}
stage("RollOut"){
input {
message "是否进行回滚"
ok "提交"
submitter "david,aa"
parameters {
choice(choices: ['yes','no'], name: 'opts')
}
}
steps{
script{
switch ("${opts}"){
case "yes":
sh " kubectl rollout undo deployment/${env.appName} -n ${env.namespace}"
break
case "no":
break
}
}
}
}
}
}
![](https://file.jishuzhan.net/article/1737941699642003457/439e909edda960af50fd2d034da26d7a.webp)
(5)观察pod变化
![](https://file.jishuzhan.net/article/1737941699642003457/4d9eab81a2a2e7db69c757c9d3d205c7.webp)
(6) 外部测试访问(当前版本为1.1.6)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/6c3aba23d9b27e633fe27bfd39ef16dd.webp)
(6)Blue Ocean弹出选项,选择yes
![](https://file.jishuzhan.net/article/1737941699642003457/3a5d5c4c0329224b198873e23f89c5e1.webp)
(7) 观察pod变化
![](https://file.jishuzhan.net/article/1737941699642003457/0ca721f9587c241ba51d6b1ac51e3122.webp)
(8)外部测试访问(当前版本回滚为1.1.5)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/e745d2c06a41d12f54e055855c78b151.webp)
8. Jenkins创建前端项目CD流水线
(1)Jenins新建前端项目CD流水线
![](https://file.jishuzhan.net/article/1737941699642003457/561a5cbf38c72013afa6560e6802addb.webp)
(2)去除其他参数,修改脚本路径
![](https://file.jishuzhan.net/article/1737941699642003457/327b82158a9420c488131554633b3ace.webp)
9. GitLab修改代码
(1)GitLab共享库新建K8S CD流水线
![](https://file.jishuzhan.net/article/1737941699642003457/ca5455b40e13558266b22c8cb178842e.webp)
(2)将之前测试成功的代码复制过来
![](https://file.jishuzhan.net/article/1737941699642003457/98e1e0906d8afa981c8aa0d195a42440.webp)
(3)GitLab前端项目新建分支RELEASE-1.1.7
![](https://file.jishuzhan.net/article/1737941699642003457/13de448e722a390555d2bfcca0aa70bc.webp)
10. Jenkins手动构建前端项目CI流水线
(1)Jenkins手动构建,分支输入RELEASE-1.1.7
![](https://file.jishuzhan.net/article/1737941699642003457/6dd86faca315b8f953036a09b2bc21bb.webp)
(2)Jenkins添加视图
![](https://file.jishuzhan.net/article/1737941699642003457/fe2b298a24ca964ffb3f20101e1022d2.webp)
(3)查看CI和CD视图
![](https://file.jishuzhan.net/article/1737941699642003457/719734601fab01daa881ccae9634265a.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/a8c4e2ae8de67d940bad7ed4057a66b8.webp)
(4)成功
![](https://file.jishuzhan.net/article/1737941699642003457/fad66a5d16c06afc0352082d9396d28d.webp)
(5)Blue Ocean 查看
![](https://file.jishuzhan.net/article/1737941699642003457/b782b2f367d99d6e9e71bdf4a9648bba.webp)
(6)Harbor查看镜像
bash
192.168.204.15/devops03/devops03-devops-ui:RELEASE-1.1.7
![](https://file.jishuzhan.net/article/1737941699642003457/e44012f2ea3987839057b9725b41c1c8.webp)
(7)Build Now 直接构建
![](https://file.jishuzhan.net/article/1737941699642003457/619588746750770ff6b128b4b053afb9.webp)
(8)观察pod变化
![](https://file.jishuzhan.net/article/1737941699642003457/037285ee512f113803ef15bb64b22a19.webp)
(9) 外部测试访问(版本升级为1.1.6)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/3e9cd4feb5195f51a6ed44268ef7ce03.webp)
11. K8S node节点连接Harbor拉取镜像
(1)拉取镜像
bash
# docker login -u admin -p Harbor12345 192.168.204.15
# docker pull 192.168.204.15/devops03/devops03-devops-ui:RELEASE-1.1.7
# docker logout 192.168.204.15
(2)node1 节点
![](https://file.jishuzhan.net/article/1737941699642003457/dde54a443fcbd3669c4727cc56f58a8f.webp)
(3)node2节点
![](https://file.jishuzhan.net/article/1737941699642003457/0f4557e4c196f114a1b4531860a2fbe9.webp)
12.Jenkins通过GitLab共享库实现前端项目在K8S自动发布应用
(1)GitLab修改部署文件
![](https://file.jishuzhan.net/article/1737941699642003457/cec190a5dbecf915d69ee3a69565c950.webp)
修改前:
![](https://file.jishuzhan.net/article/1737941699642003457/2a5734074f2a27078186fa0e29f8feb1.webp)
修改后:
![](https://file.jishuzhan.net/article/1737941699642003457/d10d901f4b51a91e317b700fb2e4e5c5.webp)
(2)Build Now 直接构建
![](https://file.jishuzhan.net/article/1737941699642003457/619588746750770ff6b128b4b053afb9.webp)
(3)观察pod变化
![](https://file.jishuzhan.net/article/1737941699642003457/8ce85ac094ff1636da93384cb7e9f46a.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/ab036c6f642a24464bbefe19e79f1744.webp)
(4) 外部测试访问(当前版本为1.1.7)
bash
# curl http://devops03-devops-ui.devops.com:31291
![](https://file.jishuzhan.net/article/1737941699642003457/f5359dc93ec698140ec516acc9f4a49e.webp)
(5)Blue Ocean弹出界面,选择no (不进行回滚)
![](https://file.jishuzhan.net/article/1737941699642003457/7ee00a6a16b84c55bddd0e911f68102b.webp)
(6) 完成
![](https://file.jishuzhan.net/article/1737941699642003457/c9f1f59410491bced37e630ee86b79bc.webp)
二、问题
1. jenkins从节点配置页面显示端口被禁用
(1)报错
![](https://file.jishuzhan.net/article/1737941699642003457/d5562aa0141a9b4e8511ffa03283eb0d.webp)
(2)原因分析
未启用WebSocket
(3) 解决方法
启用WebSocket
2.将K8S master节点配置为jenkins从节点报错
(1)报错
![](https://file.jishuzhan.net/article/1737941699642003457/bfb59611ab4eaddd5ccaea9c3b5d98f5.webp)
(2)原因分析
bash
根据报错信息可以得知,报错的原因是由于编译这份java代码的版本和当前用于运行的java版本并不一致。并且编译的版本仅支持52.0或以上的运行版本。从而可以得知,当前用于运行的java版本低于52.0。
而首先需要了解,什么是编译的版本,什么是运行的版本。需要区分开jre和jdk。
在安装java环境的时候,一个版本会分别对应两个文件夹,分别是jre和jdk。而jre表示的是java runtime environment ,即java的运行环境,而jdk表示的是Java SE Development kit(JDK),即java的编译环境。
java -version查看的是jre的版本(运行环境)。
javac -version查看的是jdk的版本(编译环境)。
(3)解决方法
bash
在查看版本后发现不一致,那么就去重新配置一下环境,保证版本一致后重新编译运行即可。
需要说明的是,我这里这份代码并不是我编写的,并且也只拿到了编译完成后的class文件,因此只能通过解决更换运行的java版本解决这个问题。
而从报错信息中可以知道,这份代码在编译后,仅支持用52.0及以上的java版本的jre环境运行,即使不知道52.0的版本对应的是哪个java版本,但肯定是需要高于1.8.0的。并且为了避免以后产生问题,我计划直接更新整个java的版本,即将jdk和jre换成一个新的版本,也考虑到以后可能还需要用到1.8.0的版本,因此并不会把1.8.0的版本删除。
① 查询java版本
bash
# java -version
![](https://file.jishuzhan.net/article/1737941699642003457/e32d1a62b3f562048443d64954e44269.webp)
②本机自带jdk 先卸载jdk
bash
# rpm -qa|grep java
# rpm -e --nodeps xxxxx
查询
![](https://file.jishuzhan.net/article/1737941699642003457/9627cca99e974c4c3e585864d4a33804.webp)
删除
![](https://file.jishuzhan.net/article/1737941699642003457/a0be64ec58b477a7c649de957b740fb6.webp)
再次查询
![](https://file.jishuzhan.net/article/1737941699642003457/6742856c69646fd5a09bd63df121c195.webp)
③ 新版的Jenkins需要安装JDK11的JAVA环境,所以我们首先要配置JDK
Java Archive Downloads - Java SE 11 | Oracle 中国
bash
#下载完之后拖到opt目录下
cd /opt
#安装JDK11
rpm -ivh jdk-11.0.15.1_linux-x64_bin.rpm
#配置环境变量
vim /etc/profile
export JAVA_HOME=/usr/java/jdk-11.0.15.1
export PATH=$PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin
#刷新配置
source /etc/profile
#查看java版本
java -version
④ ssh-copy-id 被控端IP 需要输入对应主节的root密码
bash
# ssh-copy-id root@192.168.204.180
![](https://file.jishuzhan.net/article/1737941699642003457/aaafe5eb59cb813a94792fc00334abab.webp)
⑤ 运行脚本
![](https://file.jishuzhan.net/article/1737941699642003457/4924d5700ba13d18e2304bac822cc7e5.webp)
![](https://file.jishuzhan.net/article/1737941699642003457/99a6883e254bb86d01038d5305aef2b1.webp)
⑥ 成功添加
![](https://file.jishuzhan.net/article/1737941699642003457/65b03d4a7292414e3641278a3157a8d6.webp)