一、Pipeline介绍
pipeline是jenkins官方提供一套项目构建插件,它可以用来在jenkins中实现和集成连续交付。
pipeline是一个流程,这个流程定义了一个CI/CD流程的完整步骤,通过执行这个流程代替手工自动去完成CI/CD,这个流程是由使用者自己定义的。

1.1、 Pipeline脚本语法介绍
编写时注意中英文符号
bash
node ('slave节点名') { #被操控Jenkins节点服务器
def 变量 #def可以进行变量声明
stage('阶段名A') { #流水线阶段一
执行步骤A
执行步骤B
执行步骤C
}
stage('阶段名B') { #流水线阶段二
执行步骤A
执行步骤B
执行步骤C
}
stage('阶段名C') { #流水线阶段三
执行步骤A
执行步骤B
执行步骤C
}
}
1.2 、创建Pipeline流水线测试项目

自己编写流水线格式内容



1 .3 、利用Pipeline Syntax编写Pipeline Script并构建

bash
进入Pipeline Syntax
node {
stage('clone'){
echo "git clone"
}
stage('build'){
echo "maven build"
}
stage('deploy'){
echo "project deploy"
}
stage('test'){
echo "project test"
}
}

通过脚本代码生成器,生成Pipeline脚本代码


git@192.168.200.111:/home/git/repos/app.git
将生成的代码复制到流水线脚本相应步骤的stage函数里

开始构建Pipeline项目

1 .4 、从远程仓库获取Pipeline Script并构建
在Git服务器上创建一个存放Pipeline脚本的仓库
bash
[root@Git ~]# su - git
[git@Git ~]$ cd /home/git/repos/
[git@Git repos]$ mkdir jenkinsfile.git #创建存放Pipeline脚本的仓库
[git@Git repos]$ cd jenkinsfile.git
[git@Git jenkinsfile.git]$ git --bare init
初始化空的 Git 版本库于 /home/git/repos/jenkinsfile.git/
从jenkins服务器上往远程仓库提交一个Pipeline脚本。
bash
[root@Jenkins ~]# cd test/
[root@Jenkins test]# git clone git@192.168.200.111:/home/git/repos/jenkinsfile.git
[root@Jenkins test]# cd jenkinsfile/
[root@Jenkins jenkinsfile]# mkdir itemA
[root@Jenkins jenkinsfile]# vim itemA/jenkinsfile
node {
stage('clone') {
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [],
userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/app.git']]])
}
stage('build') {
echo "maven build"
}
stage('deploy') {
echo "project deploy"
}
stage('test') {
echo "project test"
}
}
[root@Jenkins jenkinsfile]# git add *
[root@Jenkins jenkinsfile]# git commit -m "第一次提交"
[root@Jenkins jenkinsfile]# git push -u origin master
在项目中指定脚本位置


二、基于Pipeline发布PHP博客项目

|-----------------|-----------------|------------|
| 主机名 | IP地址 | 备注 |
| Git | 192.168.200.111 | Git服务器 |
| Jenkins | 192.168.200.112 | Jenkins服务器 |
| Web | 192.168.200.113 | WEB服务器 |
创建一个Pipeline流水线项目并进行参数化构建

由于我们仍旧打算将pipeline脚本放在远程Git仓库里,因此我们需要从远程Git仓库拉取Pipeline脚本,所以不选择参数化构建不支持Git的参数化。我们使用字符结构的参数化构建。


下载用于自动化发布的wordpress源码包,并上传远程git仓库
在Git服务器上创建用于存放源码的Git仓库
bash
[git@Git ~]$ cd /home/git/repos/
[git@Git repos]$ mkdir wordpress.git
[git@Git repos]$ cd wordpress.git
[git@Git wordpress.git]$ git --bare init
初始化空的 Git 版本库于 /home/git/repos/wordpress.git/
在Jenkins服务器上,克隆创建好的远程Git仓库
bash
[root@Jenkins ~]# mkdir /php-test
[root@Jenkins ~]# cd /php-test
[root@Jenkins php-test]# git clone git@192.168.200.111:/home/git/repos/wordpress.git
正克隆到 'wordpress'...
warning: 您似乎克隆了一个空版本库。
[root@Jenkins php-test]# ls
wordpress
[root@Jenkins php-test]# cd wordpress/ #手动上传WordPress源码包
[root@Jenkins php-test]# tar xf wordpress-4.9.4-zh_CN.tar.gz
[root@Jenkins php-test]# rm -rf wordpress-4.9.4-zh_CN.tar.gz
[root@Jenkins php-test]# ls
[root@Jenkins php-test]# mv wordpress/* ./
[root@Jenkins php-test]# rm -rf wordpress/
[root@Jenkins php-test]# git add *
[root@Jenkins php-test]# git commit -m "第一次提交"
[root@Jenkins php-test]# git push -u origin master
设置分布式构建的slave管理节点



Web服务器安装LNMP环境
bash
[root@localhost ~]# hostname Web
[root@localhost ~]# bash
[root@Web ~]# rpm -ivh epel-release-latest-7.noarch.rpm
[root@Web ~]# cd /etc/yum.repos.d/
[root@Web yum.repos.d]# wget http://mirrors.aliyun.com/repo/Centos-7.repo
[root@Web ~]# yum -y install nginx php-fpm php-mysql
[root@Web ~]# cd /etc/nginx/
[root@Web nginx]# cp nginx.conf{,.bak}
[root@Web nginx]# egrep -v "#|^$" nginx.conf.bak > nginx.conf
由于默认的配置文件include了/etc/nginx/conf.d/*.conf因此我们增加一个server配置文件即可
bash
[root@Web nginx]# vim /etc/nginx/conf.d/wp.conf
server {
listen 80;
server_name www.crushlinux.com;
root /usr/share/nginx/html/www.crushlinux.com;
location / {
index index.php index.html;
}
location ~\.php {
fastcgi_index index.php;
fastcgi_pass 127.0.0.1:9000;
include fastcgi.conf;
}
}
创建网页目录,并手动拉取代码模拟访问
bash
[root@Web ~]# cd /usr/share/nginx/html/
[root@Web html]# mkdir www.crushlinux.com
[root@Web html]# cd www.crushlinux.com
[root@Web www.crushlinux.com]# git clone git@192.168.200.111:/home/git/repos/wordpress.git
[root@Web www.crushlinux.com]# ls
wordpress
[root@Web www.crushlinux.com]# mv wordpress/* ./
[root@Web www.crushlinux.com]# rm -rf wordpress/
启动nginx服务和php-fpm服务
bash
[root@Web ~]# systemctl start nginx
[root@Web ~]# systemctl start php-fpm
[root@Web ~]# netstat -lnpt | grep -E "80|9000"
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 2724/php-fpm: maste
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1714/nginx: master
tcp6 0 0 :::80 :::* LISTEN 1714/nginx: master
测试主机需要先指定hosts文件,便于通过域名测试
在远程Git仓库中创建一个用于构建的Pipeline脚本
bash
[root@Jenkins ~]# rm -rf test/
[root@Jenkins ~]# mkdir test/
[root@Jenkins ~]# cd test/
[root@Jenkins test]# git clone git@192.168.200.111:/home/git/repos/jenkinsfile.git
通过流水线脚本生成器生成的阶段代码示例



bash
[root@Jenkins test]# cd jenkinsfile/
[root@Jenkins jenkinsfile]# vim itemA/jenkinsfile-php-wp
node ('php-192.168.200.113') {
stage('git checkout') {
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/wordpress.git']]])
}
stage('code copy') {
sh '''rm -rf ${WORKSPACE}/.git
[ -d /data/backup ] || mkdir -p /data/backup
mv /usr/share/nginx/html/www.crushlinux.com /data/backup/www.crushlinux.com-$(date +%F_%T)
cp -rf ${WORKSPACE} /usr/share/nginx/html/www.crushlinux.com'''
}
stage('test') {
sh 'curl http://www.crushlinux.com/status.html'
}
}
[root@Jenkins jenkinsfile]# git add *
[root@Jenkins jenkinsfile]# git commit -m "abc"
[root@Jenkins jenkinsfile]# git push -u origin master
在PHP项目代码里增加Pipeline验证用的测试页面
在项目代码里加入一个健康检查测试页面,并推送到远程Git仓库
bash
[root@Jenkins itemA]# cd /php-test/
[root@Jenkins php-test]# ls
wordpress
[root@Jenkins php-test]# cd wordpress/
[root@Jenkins wordpress]# echo "OK-version V2.0" >> status.html
将测试用页面提交到远程Git仓库
bash
[root@Jenkins wordpress]# git add *
[root@Jenkins wordpress]# git commit -m "version V2.0"
[root@Jenkins wordpress]# git push -u origin master
在web服务器做域名映射(因为要进行curl验证)
bash
[root@Web ~]# tail -1 /etc/hosts
192.168.200.113 www.crushlinux.com
浏览器访问jenkins进行PHP项目流水线发布构建


三、基于Pipeline发布JAVA博客项目

|-----------------------|-----------------|------------|
| 主机名 | IP地址 | 备注 |
| Git | 192.168.200.111 | Git服务器 |
| Jenkins | 192.168.200.112 | Jenkins服务器 |
| Web 1 | 192.168.200.113 | Tomcat服务器 |
| Web 2 | 192.168.200.114 | Tomcat服务器 |
Tomcat服务器环境部署
bash
[root@localhost ~]# hostname web1 #不同机器主机名不同
[root@localhost ~]# bash
[root@Web1 ~]#
部署jdk及环境变量
bash
[root@Web1 ~]# tar xf jdk-8u191-linux-x64.tar.gz
[root@Web1 ~]# mv jdk1.8.0_191/ /usr/local/java
[root@Web1 ~]# vim /etc/profile
export JAVA_HOME=/usr/local/java
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH
[root@Web1 ~]# rm -rf /usr/bin/java
[root@Web1 ~]# source /etc/profile
[root@Web1 ~]# ln -s /usr/local/java/bin/java /usr/local/bin/
[root@Web1 ~]# java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
部署maven及环境变量
bash
[root@Web1 ~]# tar xf apache-maven-3.5.0-bin.tar.gz
[root@Web1 ~]# mv apache-maven-3.5.0 /usr/local/maven
[root@Web1 ~]# vim /etc/profile
[root@Web1 ~]# tail -2 /etc/profile
export MAVEN_HOME=/usr/local/maven
export PATH=$PATH:${MAVEN_HOME}
[root@Web1 ~]# source /etc/profile
[root@Web1 ~]# ln -s /usr/local/maven/bin/mvn /usr/local/bin/
[root@Web1 ~]# mvn -v
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T03:39:06+08:00)
Maven home: /usr/local/maven
Java version: 1.8.0_191, vendor: Oracle Corporation
Java home: /usr/local/java/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1127.el7.x86_64", arch: "amd64", family: "unix"
部署tomcat环境变量
bash
[root@Web1 ~]# tar xf apache-tomcat-8.5.40.tar.gz
[root@Web1 ~]# mv apache-tomcat-8.5.40 /usr/local/tomcat
[root@Web1 ~]# rm -rf /usr/local/tomcat/webapps/*
[root@Web1 ~]# vim /etc/profile
export TOMCAT_HOME=/usr/local/tomcat
[root@Web1 ~]# source /etc/profile
自动部署需要解压maven构建后的war包,因此需要安装unzip
bash
[root@Web1 ~]# yum -y install unzip
自动部署需要git远程仓库代码,因此需要安装git
bash
[root@Web1 ~]# yum -y install git
构建一个B-JAVA流水线项目



下载solo项目源代码并上传Git远程仓库
bash
[root@Git ~]# su - git
[git@Git ~]$ cd /home/git/repos/
[git@Git repos]$ mkdir solo.git
[git@Git repos]$ cd solo.git
[git@Git solo.git]$ git --bare init
初始化空的 Git 版本库于 /home/git/repos/solo.git/
在jenkins服务器上进行如下操作
bash
[root@Jenkins ~]# tar xf solo.tar.gz -C /tmp/
[root@Jenkins ~]# ls /tmp/solo/
CHANGE_LOGS.html Dockerfile LICENSE pom.xml README_zh_CN.md
docker-compose.yml gulpfile.js package.json README.md src
[root@Jenkins ~]# git clone git@192.168.200.111:/home/git/repos/solo.git
正克隆到 'solo'...
warning: 您似乎克隆了一个空版本库。
[root@Jenkins ~]# mv /tmp/solo/* solo/
[root@Jenkins ~]# cd solo/
[root@Jenkins solo]# ls
CHANGE_LOGS.html Dockerfile LICENSE pom.xml README_zh_CN.md
docker-compose.yml gulpfile.js package.json README.md src
为了最后的测试,我们需要修改源代码的一个配置文件
bash
[root@Jenkins solo]# vim src/main/resources/latke.properties
29 serverHost=192.168.200.113
31 serverPort=8080
将solo源代码提交到Git仓库
bash
[root@Jenkins solo]# git add *
[root@Jenkins solo]# git commit -m "第一次提交"
[root@Jenkins solo]# git push -u origin master
创建一个流水线脚本并提交Git远程仓库
如果不出意外,构建会失败在最后一步(tomcat进程未启动)。
jenkins没办法启动tomcat的socket进程,原因在于Pipeline流水线在启动一个脚本程序的时候,这个脚本程序会运行在后台。但是当pipeline结束以后,jenkins会自动kill掉所有和pipeline有关的派生子进程。因此,我们需要在流水线脚本里加一个变量参数,不让pipeline杀掉才可以
root@Jenkins \~\]# cd test/jenkinsfile/ \[root@Jenkins jenkinsfile\]# mkdir itemB \[root@Jenkins jenkinsfile\]# cd itemB/ \[root@Jenkins itemB\]# vim jenkinsfile-java-solo node ('JAVA-192.168.200.113') { stage('git checkout') { checkout(\[$class: 'GitSCM', branches: \[\[name: '${branch}'\]\], doGenerateSubmoduleConfigurations: false, extensions: \[\], submoduleCfg: \[\], userRemoteConfigs: \[\[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4', url: 'git@192.168.200.111:/home/git/repos/solo.git'\]\]\]) } stage('maven build') { sh '/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true' } stage('code copy') { sh ''' JENKINS_NODE_COOKIE=dontkillme WWWROOT=/usr/local/tomcat/webapps/ROOT \[ -d /data/backup \] \|\| mkdir -p /data/backup if \[ -d $WWWROOT \];then mv $WWWROOT /data/backup/tomcat-$(date +%F_%T) fi unzip ${WORKSPACE}/target/\*.war -d $WWWROOT PID=$(ps -ef \| grep tomcat \| egrep -v "grep\|$$" \| awk \\'{print $2}\\') \[ -n "$PID" \] \&\& kill -9 $PID /bin/bash /usr/local/tomcat/bin/startup.sh''' } stage('test') { sh 'echo susses' } }
bash
[root@Jenkins ~]# cd test/jenkinsfile/
[root@Jenkins jenkinsfile]# mkdir itemB
[root@Jenkins jenkinsfile]# cd itemB/
[root@Jenkins itemB]# vim jenkinsfile-java-solo
node ('JAVA-192.168.200.113') {
stage('git checkout') {
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/solo.git']]])
}
stage('maven build') {
sh '/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true'
}
stage('code copy') {
sh '''
JENKINS_NODE_COOKIE=dontkillme
WWWROOT=/usr/local/tomcat/webapps/ROOT
[ -d /data/backup ] || mkdir -p /data/backup
if [ -d $WWWROOT ];then
mv $WWWROOT /data/backup/tomcat-$(date +%F_%T)
fi
unzip ${WORKSPACE}/target/*.war -d $WWWROOT
PID=$(ps -ef | grep tomcat | egrep -v "grep|$$" | awk \'{print $2}\')
[ -n "$PID" ] && kill -9 $PID
/bin/bash /usr/local/tomcat/bin/startup.sh'''
}
stage('test') {
sh 'echo susses'
}
}
推送本地仓库到远程Git
bash
[root@Jenkins jenkinsfile]# git add *
[root@Jenkins jenkinsfile]# git commit -m "java流水线脚本"
[root@Jenkins jenkinsfile]# git push -u origin master
设置分布式构建的slave管理节点


开始构建



bash
[root@Web1 ~]# netstat -lnpt | grep :8080
tcp6 0 0 :::8080 :::* LISTEN 83646/java

四、单Slave从节点SSH远程分布式构建

部署Tomcat业务环境参考之前的步骤
安装SSH Pipeline Steps插件

修改流水线脚本,并提交Git仓库
bash
[root@Jenkins itemB]# vim jenkinsfile-java-solo
node ('JAVA-192.168.200.113') {
def remote = [:]
stage('git checkout') {
sh 'hostname -I'
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/solo.git']]])
}
stage('maven build') {
sh '/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true'
}
stage('Remote SSH 200.113') {
remote.name = 'test'
remote.host = '192.168.200.113'
remote.user = 'root'
remote.password = '123456'
remote.allowAnyHosts = true
writeFile file: '113.sh',text:'echo "`hostname -I`"'
sshScript remote: remote,script: "113.sh"
}
stage('Remote SSH 200.114') {
remote.name = 'test'
remote.host = '192.168.200.114'
remote.user = 'root'
remote.password = '123456'
remote.allowAnyHosts = true
writeFile file: '114.sh',text:'echo "`hostname -I`"'
sshScript remote: remote,script: "114.sh"
}
}
[root@Jenkins itemB]# git add *
[root@Jenkins itemB]# git commit -m "java流水线脚本"
[root@Jenkins itemB]# git push -u origin master


五、多Slave从节点分布式构建

添加Slave从节点,并进行多节点构建



修改流水线脚本,并提交Git仓库
bash
[root@Jenkins itemB]# vim jenkinsfile-java-solo
node ('JAVA-192.168.200.113') {
stage('git checkout') {
sh 'hostname -I'
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/solo.git']]])
}
stage('maven build') {
sh '/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true'
}
stage('code copy') {
sh '''
JENKINS_NODE_COOKIE=dontkillme
WWWROOT=/usr/local/tomcat/webapps/ROOT
[ -d /data/backup ] || mkdir -p /data/backup
if [ -d $WWWROOT ];then
mv $WWWROOT /data/backup/tomcat-$(date +%F_%T)
fi
unzip ${WORKSPACE}/target/*.war -d $WWWROOT
PID=$(ps -ef | grep tomcat | egrep -v "grep|$$" | awk \'{print $2}\')
[ -n "$PID" ] && kill -9 $PID
/bin/bash /usr/local/tomcat/bin/startup.sh'''
}
stage('test') {
sh 'echo web1 susses'
}
}
node ('JAVA-192.168.200.114') {
stage('git checkout') {
sh 'hostname -I'
checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
doGenerateSubmoduleConfigurations: false, extensions: [],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: '1fe02564-96d3-47ef-bb7f-4ad0e72596a4',
url: 'git@192.168.200.111:/home/git/repos/solo.git']]])
}
stage('maven build') {
sh '/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true'
}
stage('code copy') {
sh '''
JENKINS_NODE_COOKIE=dontkillme
WWWROOT=/usr/local/tomcat/webapps/ROOT
[ -d /data/backup ] || mkdir -p /data/backup
if [ -d $WWWROOT ];then
mv $WWWROOT /data/backup/tomcat-$(date +%F_%T)
fi
unzip ${WORKSPACE}/target/*.war -d $WWWROOT
PID=$(ps -ef | grep tomcat | egrep -v "grep|$$" | awk \'{print $2}\')
[ -n "$PID" ] && kill -9 $PID
/bin/bash /usr/local/tomcat/bin/startup.sh'''
}
stage('test') {
sh 'echo web2 susses'
}
}
[root@Jenkins itemB]# git add *
[root@Jenkins itemB]# git commit -m "java流水线脚本"
[root@Jenkins itemB]# git push -u origin master
构建任务





