基于Jenkins+Docker的自动化部署实践——整合Git与Python脚本实现远程部署

环境说明：

Ubuntu：v24.04.1 LTS
Jekins：v2.491
Docker：v27.4.0
Gogs：v0.14.0 - 可选。可以选择Github，Gitlab或者Gitea等Git仓库，不限仓库类型
1Panel: v1.10.21-lts - 可选。这里主要用于查看和管理Docker容器

Jenkins实现参数化构建

这里通过Docker进行安装

【系统管理】【插件管理】，安装"Publish Over SSH"

【系统管理】【系统配置】，配置"SSH Servers"

填写配置信息后点击"Test Configuration"，显示"Success"说明配置成功，保存配置

新建任务

新建"test"任务，选择"流水线"

脚本编写如下：

plaintext 复制代码

pipeline {
    agent any
  
    parameters {
        string(name: 'GIT_REPO_URL', description: 'Git仓库地址', trim: true)
        string(name: 'BRANCH', defaultValue: 'main', description: '分支名称', trim: true)
        choice(name: 'DEPLOY_ENV', choices: ['deploy'], description: '部署环境')  // 改为 deploy 匹配 SSH 配置
    }
  
    environment {
        APP_NAME = 'myapp'
        IMAGE_TAG = "${BUILD_NUMBER}"
        REMOTE_DIR = '/opt'  // 修改为你配置的远程目录
    }
  
    stages {
        stage('拉取代码') {
            steps {
                git url: "${params.GIT_REPO_URL}", branch: "${params.BRANCH}"
            }
        }
      
        stage('远程构建和部署') {
            steps {
                script {
                    sshPublisher(
                        publishers: [
                            sshPublisherDesc(
                                configName: 'deploy',  // 修改为你配置的 Name
                                verbose: true,
                                transfers: [
                                    sshTransfer(
                                        sourceFiles: "**/*",
                                        remoteDirectory: "${APP_NAME}-${BUILD_NUMBER}",
                                        execCommand: """
                                            cd ${REMOTE_DIR}/${APP_NAME}-${BUILD_NUMBER}
                                          
                                            # 构建和部署
                                            docker build -t ${APP_NAME}:${IMAGE_TAG} .
                                            docker stop ${APP_NAME} || true
                                            docker rm ${APP_NAME} || true
                                            docker run -d --name ${APP_NAME} \
                                                -p 8880:5000 \
                                                --restart unless-stopped \
                                                ${APP_NAME}:${IMAGE_TAG}
                                          
                                            # 清理
                                            cd ..
                                            rm -rf ${APP_NAME}-${BUILD_NUMBER}
                                            docker system prune -f
                                        """
                                    )
                                ]
                            )
                        ]
                    )
                }
            }
        }
      
        stage('健康检查') {
            steps {
                sleep 15
                sshPublisher(
                    publishers: [
                        sshPublisherDesc(
                            configName: 'deploy',
                            transfers: [
                                sshTransfer(
                                    execCommand: '''
                                        max_attempts=5
                                        attempt=1
                                        while [ $attempt -le $max_attempts ]; do
                                            response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8880/health)
                                            if [ "$response" = "200" ]; then
                                                echo "Health check succeeded"
                                                exit 0
                                            fi
                                            echo "Attempt $attempt failed, waiting... (Status code: $response)"
                                            sleep 10
                                            attempt=$((attempt + 1))
                                        done
                                        echo "Health check failed after $max_attempts attempts"
                                        exit 1
                                    '''
                                )
                            ]
                        )
                    ]
                )
            }
        }
    }
  
    post {
        success {
            echo "部署成功: ${APP_NAME}:${IMAGE_TAG}"
        }
        failure {
            echo "部署失败"
        }
        cleanup {
            cleanWs()
        }
    }
}

打开【Build with Parameters】填写Git仓库地址和仓库分支，点击【Build】

打开【Console Output】查看构建日志

日志显示如下，代表构建完成

plaintext 复制代码

...
[Pipeline] step
SSH: Connecting from host [94e37d92d688]
SSH: Connecting with configuration [deploy] ...
SSH: EXEC: completed after 200 ms
SSH: Disconnecting configuration [deploy] ...
SSH: Transferred 0 file(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] echo
部署成功: myapp:27
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] done
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

这时在部署的主机上可以看到自动化部署的Docker容器

健康检查

由于在项目docker_net8_webapi_fortran中已经实现/health接口，所以在流水线中健康检查逻辑如下，这里支持重试机制

plaintext 复制代码

stage('健康检查') {
    steps {
        sleep 15
        sshPublisher(
            publishers: [
                sshPublisherDesc(
                    configName: 'deploy',
                    transfers: [
                        sshTransfer(
                            execCommand: '''
                                max_attempts=5
                                attempt=1
                                while [ $attempt -le $max_attempts ]; do
                                    response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8880/health)
                                    if [ "$response" = "200" ]; then
                                        echo "Health check succeeded"
                                        exit 0
                                    fi
                                    echo "Attempt $attempt failed, waiting... (Status code: $response)"
                                    sleep 10
                                    attempt=$((attempt + 1))
                                done
                                echo "Health check failed after $max_attempts attempts"
                                exit 1
                            '''
                        )
                    ]
                )
            ]
        )
    }
}

参数化配置Python脚本实现远程部署

刚刚我们通过手动填写参数的方式完成项目自动化构建，进一步，可以使用python脚本远程进行触发构建过程，并且实现参数化配置

python 复制代码

import requests
import json
import time
import yaml
import logging
import urllib3
from typing import Optional, Dict, Any
from urllib.parse import quote
from tenacity import retry, stop_after_attempt, wait_exponential

# 配置日志
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('deployment.log'),
        logging.StreamHandler()
    ]
)
logger = logging.getLogger('JenkinsDeployer')

class Config:
    def __init__(self, config_file: str = "config.yaml"):
        with open(config_file, 'r', encoding='utf-8') as f:
            self.config = yaml.safe_load(f)
      
        # Jenkins配置
        self.jenkins = self.config['jenkins']
        self.deployment = self.config['deployment']

class JenkinsDeployer:
    def __init__(self, config: Config):
        self.config = config
        self.JENKINS_URL = config.jenkins['url']
        self.USER = config.jenkins['user']
        self.API_TOKEN = config.jenkins['token']
        self.JOB_NAME = config.jenkins['job_name']

    @retry(
        stop=stop_after_attempt(3),
        wait=wait_exponential(multiplier=1, min=4, max=10)
    )
    def _make_request(self, method: str, url: str, **kwargs) -> requests.Response:
        """通用的请求方法，带重试机制"""
        logger.info(f"发送 {method} 请求到 {url}")
        response = requests.request(
            method,
            url,
            auth=(self.USER, self.API_TOKEN),
            timeout=30,
            verify=False,
            **kwargs
        )
        response.raise_for_status()
        return response

    def trigger_deploy(self, 
                      git_repo_url: str, 
                      branch: str = "main", 
                      deploy_env: str = "deploy",
                      wait: bool = True) -> Dict[str, Any]:
        """触发Jenkins部署"""
        try:
             # 参数不进行 URL 编码
            params = {
                "GIT_REPO_URL": git_repo_url,
                "BRANCH": branch, 
                "DEPLOY_ENV": deploy_env
            }
          
            # 构建 URL
            url = f"{self.JENKINS_URL}/job/{self.JOB_NAME}/buildWithParameters"
          
            logger.info(f"触发构建: {url}")
            logger.info(f"参数: {json.dumps(params, indent=2, ensure_ascii=False)}")
          
            # 发送请求
            response = self._make_request('POST', url, params=params)
          
            if response.status_code == 201:
                queue_url = response.headers.get('Location')
                if not queue_url:
                    return {"error": "未获取到队列URL"}
              
                logger.info(f"构建队列 URL: {queue_url}")
                build_number = self._get_build_number(queue_url)
              
                if build_number:
                    logger.info(f"部署已触发，构建号: {build_number}")
                  
                    if wait:
                        return self.wait_for_completion(build_number)
                    return {"build_number": build_number, "status": "STARTED"}
                else:
                    return {"error": "未能获取构建号"}
          
            return {
                "error": f"触发失败: {response.status_code}",
                "details": response.text
            }
          
        except Exception as e:
            logger.error(f"部署触发失败: {str(e)}", exc_info=True)
            return {"error": f"部署触发失败: {str(e)}"}

    def _get_build_number(self, queue_url: str) -> Optional[int]:
        """获取构建号"""
        max_attempts = 10
        for attempt in range(max_attempts):
            try:
                response = self._make_request('GET', f"{queue_url}api/json")
              
                logger.info(f"尝试获取构建号 ({attempt + 1}/{max_attempts})")
                logger.debug(f"队列响应: {response.text}")
              
                data = response.json()
                if "executable" in data and "number" in data["executable"]:
                    return data["executable"]["number"]
                elif "why" in data:
                    logger.info(f"构建等待中: {data['why']}")
                  
                time.sleep(2)
            except Exception as e:
                logger.error(f"获取构建号失败 ({attempt + 1}/{max_attempts}): {e}")
        return None

    def get_build_status(self, build_number: int) -> Dict[str, Any]:
        """获取构建状态"""
        url = f"{self.JENKINS_URL}/job/{self.JOB_NAME}/{build_number}/api/json"
        try:
            response = self._make_request('GET', url)
            build_info = response.json()
          
            return {
                "number": build_info["number"],
                "result": build_info.get("result", "IN_PROGRESS"),
                "url": build_info["url"],
                "duration": build_info["duration"],
                "timestamp": build_info["timestamp"]
            }
        except Exception as e:
            logger.error(f"获取构建状态失败: {e}", exc_info=True)
            return {"error": f"获取状态失败: {str(e)}"}

    def wait_for_completion(self, build_number: int, timeout: int = 300) -> Dict[str, Any]:
        """等待部署完成"""
        start_time = time.time()
        while time.time() - start_time < timeout:
            status = self.get_build_status(build_number)
          
            if "error" in status:
                return status
          
            if status["result"] and status["result"] != "IN_PROGRESS":
                return status
          
            logger.info(f"部署进行中... ({int(time.time() - start_time)}s)")
            time.sleep(10)
          
        return {"error": "部署超时"}

    def check_deployment_health(self, host: str, port: int, max_attempts: int = 5) -> bool:
        """检查部署的应用是否健康"""
        health_url = f"http://{host}:{port}/health"
      
        for attempt in range(max_attempts):
            try:
                response = requests.get(health_url, timeout=5)
                if response.status_code == 200:
                    logger.info(f"健康检查成功 (尝试 {attempt + 1}/{max_attempts})")
                    return True
            except Exception as e:
                logger.warning(f"健康检查失败 (尝试 {attempt + 1}/{max_attempts}): {e}")
          
            if attempt < max_attempts - 1:
                time.sleep(10)
      
        return False

def main():
    # 禁用 SSL 警告
    urllib3.disable_warnings()
  
    try:
        # 初始化配置
        config = Config("config.yaml")
        deployer = JenkinsDeployer(config)
      
        # 触发部署
        result = deployer.trigger_deploy(
            git_repo_url=config.deployment['git_repo'],
            branch=config.deployment['branch'],
            deploy_env="deploy",
            wait=True
        )
      
        if "error" in result:
            logger.error(f"部署失败: {result['error']}")
            return
      
        # 执行健康检查
        health_config = config.deployment['health_check']
        is_healthy = deployer.check_deployment_health(
            host=health_config['host'],
            port=health_config['port'],
            max_attempts=health_config['max_attempts']
        )
      
        if is_healthy:
            logger.info("部署成功且应用程序运行正常")
        else:
            logger.warning("部署可能成功但健康检查失败")
          
    except Exception as e:
        logger.error(f"部署过程出错: {str(e)}", exc_info=True)

if __name__ == "__main__":
    main()

config.yaml配置文件如下：

plaintext 复制代码

jenkins:
  url: "http://192.168.1.140:8563"
  user: "admin"
  token: "1144e4584a109badf5051a42a960aef11d"
  job_name: "test"

deployment:
  git_repo: "http://192.168.1.140:10880/root/docker_net8_webapi_fortran.git"
  branch: "master"
  health_check:
    host: "192.168.1.140"
    port: 8880
    max_attempts: 5

运行python脚本，日志结果如下：

plaintext 复制代码

$ python .\main.py
2024-12-24 10:22:06,538 - JenkinsDeployer - INFO - 触发构建: http://192.168.1.140:8563/job/test/buildWithParameters
2024-12-24 10:22:06,539 - JenkinsDeployer - INFO - 参数: {
  "GIT_REPO_URL": "http://192.168.1.140:10880/root/docker_net8_webapi_fortran.git",
  "BRANCH": "master",
  "DEPLOY_ENV": "deploy"
}
2024-12-24 10:22:06,539 - JenkinsDeployer - INFO - 发送 POST 请求到 http://192.168.1.140:8563/job/test/buildWithParameters
2024-12-24 10:22:06,636 - JenkinsDeployer - INFO - 构建队列 URL: http://192.168.1.140:8563/queue/item/62/
2024-12-24 10:22:06,637 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/queue/item/62/api/json
2024-12-24 10:22:06,788 - JenkinsDeployer - INFO - 尝试获取构建号 (1/10)
2024-12-24 10:22:06,789 - JenkinsDeployer - INFO - 构建等待中: In the quiet period. Expires in 4.8 sec
2024-12-24 10:22:08,790 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/queue/item/62/api/json
2024-12-24 10:22:08,921 - JenkinsDeployer - INFO - 尝试获取构建号 (2/10)
2024-12-24 10:22:08,921 - JenkinsDeployer - INFO - 构建等待中: In the quiet period. Expires in 2.7 sec
2024-12-24 10:22:10,929 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/queue/item/62/api/json
2024-12-24 10:22:11,029 - JenkinsDeployer - INFO - 尝试获取构建号 (3/10)
2024-12-24 10:22:11,029 - JenkinsDeployer - INFO - 构建等待中: In the quiet period. Expires in 0.6 sec
2024-12-24 10:22:13,032 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/queue/item/62/api/json
2024-12-24 10:22:13,207 - JenkinsDeployer - INFO - 尝试获取构建号 (4/10)
2024-12-24 10:22:13,207 - JenkinsDeployer - INFO - 部署已触发，构建号: 26
2024-12-24 10:22:13,208 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/job/test/26/api/json
2024-12-24 10:22:13,357 - JenkinsDeployer - INFO - 部署进行中... (0s)
2024-12-24 10:22:23,364 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/job/test/26/api/json
2024-12-24 10:22:23,640 - JenkinsDeployer - INFO - 部署进行中... (10s)
2024-12-24 10:22:33,644 - JenkinsDeployer - INFO - 发送 GET 请求到 http://192.168.1.140:8563/job/test/26/api/json
2024-12-24 10:22:33,907 - JenkinsDeployer - INFO - 健康检查成功 (尝试 1/5)
2024-12-24 10:22:33,908 - JenkinsDeployer - INFO - 部署成功且应用程序运行正常

基于Jenkins+Docker的自动化部署实践——整合Git与Python脚本实现远程部署

Jenkins实现参数化构建

新建任务

健康检查

参数化配置Python脚本实现远程部署

参考