Python 在 Jenkins Pipeline 中的使用总结

kaico20182026-06-01 9:18

Python 在 Jenkins Pipeline 中的使用总结

目标：掌握在 Jenkins Pipeline 中使用 Python 脚本处理文件、调用 API、发送通知、生成报告、管理配置等常见运维场景，提升 CI/CD 流程的自动化能力

一、Pipeline 中运行 Python 的方式

1.1 直接执行 Python 脚本

最基本的方式：将 Python 脚本放在仓库中，Pipeline 通过 sh 调用。

groovy 复制代码 
pipeline {
    agent any
    stages {
        stage('Run Python') {
            steps {
                sh 'python3 scripts/my_script.py'
            }
        }
    }
}

1.2 内联 Python 代码

使用三引号在 Pipeline 中直接嵌入 Python 代码，适合简单逻辑，无需额外文件。

groovy 复制代码 
pipeline {
    agent any
    stages {
        stage('Inline Python') {
            steps {
                sh '''
                    python3 -c "
import json
data = {'version': '1.0.0', 'env': 'staging'}
print(json.dumps(data, indent=2))
                    "
                '''
            }
        }
    }
}

1.3 使用 Python 虚拟环境

为每个构建创建隔离的虚拟环境，避免依赖冲突，适合需要安装第三方库的场景。

groovy 复制代码 
pipeline {
    agent any
    stages {
        stage('Setup Venv & Run') {
            steps {
                sh '''
                    python3 -m venv .venv
                    . .venv/bin/activate
                    pip install requests python-gitlab jinja2
                    python3 scripts/main.py
                '''
            }
        }
    }
}

1.4 使用 requirements.txt 管理依赖

将 Python 依赖声明在 requirements.txt 中，构建时统一安装，确保环境可复现。

groovy 复制代码 
pipeline {
    agent any
    stages {
        stage('Install & Run') {
            steps {
                sh '''
                    python3 -m venv .venv
                    . .venv/bin/activate
                    pip install -r scripts/requirements.txt
                    python3 scripts/main.py
                '''
            }
        }
    }
}
复制代码 
# scripts/requirements.txt
requests>=2.28.0
python-gitlab>=3.14.0
jinja2>=3.1.0
pyyaml>=6.0
python-jenkins>=1.8.0

1.5 使用 Docker Agent 运行 Python

通过 Docker 镜像提供一致的 Python 运行环境，避免 Jenkins 节点环境差异。

groovy 复制代码 
pipeline {
    agent {
        docker {
            image 'python:3.11-slim'
            args '-v $HOME/.cache/pip:/root/.cache/pip'
        }
    }
    stages {
        stage('Run in Docker') {
            steps {
                sh 'pip install -r scripts/requirements.txt'
                sh 'python3 scripts/main.py'
            }
        }
    }
}

1.6 Pipeline 与 Python 的数据传递

Pipeline 变量与 Python 脚本之间的双向数据传递，是实际使用中最核心的技巧。

groovy 复制代码 
pipeline {
    agent any
    environment {
        APP_VERSION = '1.2.0'
        DEPLOY_ENV = 'staging'
    }
    stages {
        stage('Pass Vars to Python') {
            steps {
                // 方式1：通过环境变量传递（推荐）
                sh '''
                    export APP_VERSION="${APP_VERSION}"
                    export DEPLOY_ENV="${DEPLOY_ENV}"
                    python3 scripts/process.py
                '''

                // 方式2：通过命令行参数传递
                sh "python3 scripts/process.py --version ${APP_VERSION} --env ${DEPLOY_ENV}"

                // 方式3：Python 输出写文件，Pipeline 读取
                sh 'python3 scripts/get_version.py > version.txt'
                script {
                    def version = readFile('version.txt').trim()
                    echo "Python 返回的版本: ${version}"
                    env.RESOLVED_VERSION = version
                }
            }
        }

        stage('Python Return Value') {
            steps {
                script {
                    // 捕获 Python 脚本的标准输出
                    def result = sh(script: 'python3 -c "print(1+1)"', returnStdout: true).trim()
                    echo "Python 计算结果: ${result}"

                    // 捕获退出码
                    def exitCode = sh(script: 'python3 scripts/check.py', returnStatus: true)
                    if (exitCode != 0) {
                        echo "检查未通过，退出码: ${exitCode}"
                    }
                }
            }
        }
    }
}
python 复制代码 
# scripts/process.py
import os

app_version = os.environ.get('APP_VERSION', 'unknown')
deploy_env = os.environ.get('DEPLOY_ENV', 'dev')
print(f"处理版本: {app_version}, 环境: {deploy_env}")

二、文件处理

2.1 解析与修改配置文件

在部署流程中，经常需要根据环境动态修改配置文件（YAML、JSON、INI、.env 等）。

YAML 配置处理
python 复制代码 
# scripts/config_processor.py
import yaml
import os
import sys

def update_k8s_deployment(yaml_file, image_tag, replicas):
    with open(yaml_file, 'r', encoding='utf-8') as f:
        docs = list(yaml.safe_load_all(f))

    for doc in docs:
        if doc.get('kind') == 'Deployment':
            doc['spec']['replicas'] = replicas
            for container in doc['spec']['template']['spec']['containers']:
                container['image'] = f"myapp:{image_tag}"
        if doc.get('kind') == 'ConfigMap':
            doc['data']['APP_VERSION'] = image_tag
            doc['data']['DEPLOY_ENV'] = os.environ.get('DEPLOY_ENV', 'dev')

    with open(yaml_file, 'w', encoding='utf-8') as f:
        yaml.dump_all(docs, f, default_flow_style=False, allow_unicode=True)

    print(f"配置已更新: image={image_tag}, replicas={replicas}")

if __name__ == '__main__':
    update_k8s_deployment(
        yaml_file=sys.argv[1],
        image_tag=sys.argv[2],
        replicas=int(sys.argv[3])
    )
groovy 复制代码 
// Jenkinsfile
stage('Update K8s Config') {
    steps {
        sh """
            python3 scripts/config_processor.py \
                k8s/deployment.yaml \
                ${env.BUILD_NUMBER} \
                3
        """
    }
}
JSON 配置处理
python 复制代码 
# scripts/update_package_json.py
import json
import os

def update_version(package_json_path, new_version):
    with open(package_json_path, 'r', encoding='utf-8') as f:
        data = json.load(f)

    data['version'] = new_version

    if 'dependencies' in data:
        for dep in data['dependencies']:
            if data['dependencies'][dep].startswith('^') or data['dependencies'][dep].startswith('~'):
                prefix = data['dependencies'][dep][0]
                data['dependencies'][dep] = f"{prefix}{new_version}"

    with open(package_json_path, 'w', encoding='utf-8') as f:
        json.dump(data, f, indent=2, ensure_ascii=False)

    print(f"版本已更新为: {new_version}")

if __name__ == '__main__':
    update_version('package.json', os.environ['NEW_VERSION'])
.env 文件处理
python 复制代码 
# scripts/update_env.py
import os

def update_env_file(env_file, updates):
    lines = []
    updated_keys = set()

    if os.path.exists(env_file):
        with open(env_file, 'r', encoding='utf-8') as f:
            for line in f:
                stripped = line.strip()
                if '=' in stripped and not stripped.startswith('#'):
                    key = stripped.split('=', 1)[0].strip()
                    if key in updates:
                        lines.append(f"{key}={updates[key]}\n")
                        updated_keys.add(key)
                        continue
                lines.append(line)

    for key, value in updates.items():
        if key not in updated_keys:
            lines.append(f"{key}={value}\n")

    with open(env_file, 'w', encoding='utf-8') as f:
        f.writelines(lines)

if __name__ == '__main__':
    update_env_file('.env', {
        'APP_VERSION': os.environ.get('APP_VERSION', 'latest'),
        'DEPLOY_ENV': os.environ.get('DEPLOY_ENV', 'dev'),
        'BUILD_NUMBER': os.environ.get('BUILD_NUMBER', '0'),
    })

2.2 日志文件分析与过滤

解析构建日志、测试报告，提取关键信息，决定流水线后续行为。

python 复制代码 
# scripts/log_analyzer.py
import re
import sys
import json

def analyze_build_log(log_file):
    errors = []
    warnings = []
    test_summary = {}

    with open(log_file, 'r', encoding='utf-8', errors='ignore') as f:
        for line_num, line in enumerate(f, 1):
            if re.search(r'ERROR|CRITICAL|FATAL', line, re.IGNORECASE):
                errors.append({"line": line_num, "content": line.strip()})
            elif re.search(r'WARNING|WARN', line, re.IGNORECASE):
                warnings.append({"line": line_num, "content": line.strip()})

            match = re.search(r'(\d+) passed.*?(\d+) failed.*?(\d+) skipped', line)
            if match:
                test_summary = {
                    "passed": int(match.group(1)),
                    "failed": int(match.group(2)),
                    "skipped": int(match.group(3))
                }

    result = {
        "error_count": len(errors),
        "warning_count": len(warnings),
        "errors": errors[:10],
        "warnings": warnings[:10],
        "test_summary": test_summary
    }

    print(json.dumps(result, indent=2, ensure_ascii=False))

    if len(errors) > 0:
        sys.exit(1)

if __name__ == '__main__':
    analyze_build_log(sys.argv[1])
groovy 复制代码 
stage('Analyze Log') {
    steps {
        script {
            def result = sh(
                script: 'python3 scripts/log_analyzer.py build.log 2>&1',
                returnStdout: true
            ).trim()
            def analysis = readJSON(text: result)
            echo "错误数: ${analysis.error_count}, 警告数: ${analysis.warning_count}"
        }
    }
}

2.3 文件批量处理与模板渲染

使用 Jinja2 模板引擎批量生成配置文件，适合多环境、多服务的配置管理。

python 复制代码 
# scripts/render_configs.py
from jinja2 import Environment, FileSystemLoader
import os
import json

def render_configs(template_dir, output_dir, env_vars):
    env = Environment(
        loader=FileSystemLoader(template_dir),
        keep_trailing_newline=True
    )

    context = {
        'app_name': os.environ.get('APP_NAME', 'myapp'),
        'version': os.environ.get('APP_VERSION', 'latest'),
        'env': os.environ.get('DEPLOY_ENV', 'dev'),
        'replicas': int(os.environ.get('REPLICAS', '2')),
        'db_host': os.environ.get('DB_HOST', 'localhost'),
        'db_port': os.environ.get('DB_PORT', '3306'),
        'redis_url': os.environ.get('REDIS_URL', 'redis://localhost:6379'),
    }
    context.update(env_vars)

    os.makedirs(output_dir, exist_ok=True)

    for template_name in env.list_templates():
        if template_name.endswith(('.j2', '.jinja2', '.tpl')):
            template = env.get_template(template_name)
            output_name = template_name.replace('.j2', '').replace('.jinja2', '').replace('.tpl', '')
            output_path = os.path.join(output_dir, output_name)

            os.makedirs(os.path.dirname(output_path), exist_ok=True)
            with open(output_path, 'w', encoding='utf-8') as f:
                f.write(template.render(**context))

            print(f"已生成: {output_path}")

if __name__ == '__main__':
    extra_vars = json.loads(os.environ.get('EXTRA_VARS', '{}'))
    render_configs('templates/', 'output/', extra_vars)

模板示例 (templates/config.yaml.j2)：

yaml 复制代码 
app:
  name: {{ app_name }}
  version: {{ version }}
  environment: {{ env }}

server:
  host: 0.0.0.0
  port: 8000
  workers: {{ replicas }}

database:
  host: {{ db_host }}
  port: {{ db_port }}
  name: {{ app_name }}_{{ env }}

redis:
  url: {{ redis_url }}

三、调用 GitLab API

3.1 GitLab REST API 速查表

以下表格汇总了 Pipeline 中最常用的 GitLab REST API，按功能分类，包含请求方法、路径、参数和返回值。

认证方式 ：所有请求在 Header 中携带 PRIVATE-TOKEN: <your_token>，或使用 URL 参数 ?private_token=<your_token>

基础路径 ：{GITLAB_URL}/api/v4

3.1.1 项目（Projects）
功能 方法 路径 请求参数 返回值（关键字段）
获取项目信息 GET /projects/{id} id: 项目ID或URL编码路径 id, name, path_with_namespace, default_branch, web_url
搜索项目 GET /projects search: 搜索关键词; membership: bool; simple: bool 项目列表，每项含 id, name, path_with_namespace
获取项目文件 GET /projects/{id}/repository/files/{file_path} ref: 分支名或Tag; file_path: URL编码的文件路径 file_name, file_path, content(Base64), ref, blob_id
创建项目文件 POST /projects/{id}/repository/files/{file_path} branch, content, commit_message, encoding(text/base64) file_path, branch
更新项目文件 PUT /projects/{id}/repository/files/{file_path} branch, content, commit_message, encoding file_path, branch
删除项目文件 DELETE /projects/{id}/repository/files/{file_path} branch, commit_message 204 No Content
3.1.2 合并请求（Merge Requests）
功能 方法 路径 请求参数 返回值（关键字段）
列出合并请求 GET /projects/{id}/merge_requests state: opened/closed/merged/all; scope: created_by_me/assigned_to_me/all; target_branch MR列表，每项含 iid, title, source_branch, target_branch, state, web_url, author
获取单个MR GET /projects/{id}/merge_requests/{mr_iid} 无 iid, title, description, source_branch, target_branch, state, merge_status, web_url
创建合并请求 POST /projects/{id}/merge_requests source_branch, target_branch , title*, description, labels, assignee_id iid, title, web_url, state
接受/合并MR PUT /projects/{id}/merge_requests/{mr_iid}/merge merge_commit_message, should_remove_source_branch, squash state: "merged", merge_commit_sha
获取MR变更 GET /projects/{id}/merge_requests/{mr_iid}/changes 无 同获取单个MR + changes: {`old_path`, `new_path`, `diff`}
添加MR评论 POST /projects/{id}/merge_requests/{mr_iid}/notes body*: 评论内容 id, body, author, created_at
关闭合并请求 PUT /projects/{id}/merge_requests/{mr_iid} state_event: close state: "closed"
3.1.3 分支与标签（Branches & Tags）
功能 方法 路径 请求参数 返回值（关键字段）
列出分支 GET /projects/{id}/repository/branches search: 搜索关键词 分支列表，每项含 name, merged, default, commit
获取分支 GET /projects/{id}/repository/branches/{branch} branch: 分支名 name, merged, commit, developers_can_push
创建分支 POST /projects/{id}/repository/branches branch: 新分支名; ref: 源分支名或SHA name, commit
删除分支 DELETE /projects/{id}/repository/branches/{branch} branch: 分支名 204 No Content
列出标签 GET /projects/{id}/repository/tags order_by: updated/name; sort: asc/desc 标签列表，每项含 name, target, message
创建标签 POST /projects/{id}/repository/tags tag_name: 标签名; ref : 源分支或SHA; message: 注释消息 name, target, message, commit
删除标签 DELETE /projects/{id}/repository/tags/{tag_name} tag_name: 标签名 204 No Content
3.1.4 Issue
功能 方法 路径 请求参数 返回值（关键字段）
列出Issue GET /projects/{id}/issues state: opened/closed/all; labels: 逗号分隔标签; assignee_id Issue列表，每项含 iid, title, state, labels, web_url
获取单个Issue GET /projects/{id}/issues/{issue_iid} 无 iid, title, description, state, labels, assignees
创建Issue POST /projects/{id}/issues title*, description, labels, assignee_ids, milestone_id iid, title, web_url, state
更新Issue PUT /projects/{id}/issues/{issue_iid} title, description, state_event: close/reopen, labels iid, title, state
添加Issue评论 POST /projects/{id}/issues/{issue_iid}/notes body*: 评论内容 id, body, author, created_at
3.1.5 Pipeline 与 Job
功能 方法 路径 请求参数 返回值（关键字段）
列出Pipeline GET /projects/{id}/pipelines status: running/pending/success/failed/canceled; ref: 分支名; sha Pipeline列表，每项含 id, status, ref, sha, web_url
获取Pipeline详情 GET /projects/{id}/pipelines/{pipeline_id} 无 id, status, ref, sha, duration, web_url
重试Pipeline POST /projects/{id}/pipelines/{pipeline_id}/retry 无 Pipeline对象
取消Pipeline POST /projects/{id}/pipelines/{pipeline_id}/cancel 无 Pipeline对象
列出Jobs GET /projects/{id}/pipelines/{pipeline_id}/jobs scope: running/pending/success/failed/canceled Job列表，每项含 id, name, status, stage, duration
获取Job日志 GET /projects/{id}/jobs/{job_id}/trace 无 纯文本日志内容
获取Job产物 GET /projects/{id}/jobs/{job_id}/artifacts 无 产物文件（zip）
3.1.6 代码搜索与提交
功能 方法 路径 请求参数 返回值（关键字段）
搜索代码 GET /projects/{id}/search scope: blobs; search*: 搜索关键词; ref: 分支名 搜索结果列表，含 filename, ref, startline, data
列出提交 GET /projects/{id}/repository/commits ref_name: 分支名; since/until: ISO8601日期; per_page 提交列表，每项含 id, short_id, title, author_name, committed_date
获取单个提交 GET /projects/{id}/repository/commits/{sha} 无 id, title, message, author_name, stats: {additions, deletions, total}
对比提交 GET /projects/{id}/repository/compare from: 源分支/SHA; to: 目标分支/SHA commits, diffs: {`old_path`, `new_path`, `diff`}

3.2 使用 python-gitlab 库

python-gitlab 是 GitLab 官方 Python 客户端，封装了完整的 REST API，推荐优先使用。

bash 复制代码 
pip install python-gitlab
python 复制代码 
# scripts/gitlab_client.py
import gitlab
import os
import json
import sys

class GitLabClient:
    def __init__(self):
        self.gl = gitlab.Gitlab(
            url=os.environ.get('GITLAB_URL', 'https://gitlab.com'),
            private_token=os.environ.get('GITLAB_TOKEN')
        )

    def get_project(self, project_id):
        return self.gl.projects.get(project_id)

    def list_merge_requests(self, project_id, state='opened'):
        project = self.get_project(project_id)
        mrs = project.mergerequests.list(state=state, all=True)
        return [{
            'iid': mr.iid,
            'title': mr.title,
            'author': mr.author['username'],
            'source_branch': mr.source_branch,
            'target_branch': mr.target_branch,
            'state': mr.state,
            'web_url': mr.web_url
        } for mr in mrs]

    def create_merge_request(self, project_id, source_branch, target_branch, title):
        project = self.get_project(project_id)
        mr = project.mergerequests.create({
            'source_branch': source_branch,
            'target_branch': target_branch,
            'title': title
        })
        return {'iid': mr.iid, 'web_url': mr.web_url}

    def accept_merge_request(self, project_id, mr_iid):
        project = self.get_project(project_id)
        mr = project.mergerequests.get(mr_iid)
        mr.merge()
        return {'state': 'merged', 'iid': mr_iid}

    def get_branches(self, project_id, search=None):
        project = self.get_project(project_id)
        branches = project.branches.list(search=search, all=True)
        return [{'name': b.name, 'merged': b.merged, 'default': b.default} for b in branches]

    def delete_branch(self, project_id, branch_name):
        project = self.get_project(project_id)
        project.branches.delete(branch_name)
        print(f"分支已删除: {branch_name}")

    def create_tag(self, project_id, tag_name, ref, message=''):
        project = self.get_project(project_id)
        tag = project.tags.create({'tag_name': tag_name, 'ref': ref, 'message': message})
        return {'name': tag.name, 'target': tag.target}

    def list_tags(self, project_id, order_by='updated', sort='desc'):
        project = self.get_project(project_id)
        tags = project.tags.list(order_by=order_by, sort=sort)
        return [{'name': t.name, 'target': t.target, 'message': t.message} for t in tags]

    def get_file(self, project_id, file_path, ref='main'):
        project = self.get_project(project_id)
        f = project.files.get(file_path=file_path, ref=ref)
        return f.decode().decode('utf-8')

    def update_file(self, project_id, file_path, content, commit_message, branch='main'):
        project = self.get_project(project_id)
        f = project.files.get(file_path=file_path, ref=branch)
        f.content = content.encode('utf-8')
        f.save(branch=branch, commit_message=commit_message)
        return {'file_path': file_path, 'branch': branch}

    def create_issue(self, project_id, title, description='', labels=None):
        project = self.get_project(project_id)
        issue = project.issues.create({
            'title': title,
            'description': description,
            'labels': labels or []
        })
        return {'iid': issue.iid, 'web_url': issue.web_url}

    def add_comment(self, project_id, mr_iid, body):
        project = self.get_project(project_id)
        mr = project.mergerequests.get(mr_iid)
        note = mr.notes.create({'body': body})
        return {'id': note.id, 'body': note.body}

if __name__ == '__main__':
    client = GitLabClient()
    action = sys.argv[1]

    if action == 'list-mr':
        project_id = sys.argv[2]
        mrs = client.list_merge_requests(project_id)
        print(json.dumps(mrs, indent=2, ensure_ascii=False))

    elif action == 'create-mr':
        project_id = sys.argv[2]
        result = client.create_merge_request(
            project_id=project_id,
            source_branch=sys.argv[3],
            target_branch=sys.argv[4],
            title=sys.argv[5]
        )
        print(json.dumps(result, indent=2, ensure_ascii=False))

    elif action == 'create-tag':
        project_id = sys.argv[2]
        result = client.create_tag(
            project_id=project_id,
            tag_name=sys.argv[3],
            ref=sys.argv[4],
            message=sys.argv[5] if len(sys.argv) > 5 else ''
        )
        print(json.dumps(result, indent=2, ensure_ascii=False))

3.3 Pipeline 中使用 GitLab API

groovy 复制代码 
pipeline {
    agent any
    environment {
        GITLAB_URL = 'https://gitlab.example.com'
        GITLAB_TOKEN = credentials('gitlab-api-token')
        PROJECT_ID = '123'
    }
    stages {
        stage('List Open MRs') {
            steps {
                sh '''
                    pip install python-gitlab 2>/dev/null
                    python3 scripts/gitlab_client.py list-mr ${PROJECT_ID}
                '''
            }
        }

        stage('Auto Create MR') {
            when {
                branch 'feature/*'
            }
            steps {
                script {
                    def branch_name = env.BRANCH_NAME
                    def mr_title = "Merge ${branch_name} into main"
                    def result = sh(
                        script: """
                            python3 scripts/gitlab_client.py create-mr \
                                ${PROJECT_ID} \
                                ${branch_name} \
                                main \
                                "${mr_title}"
                        """,
                        returnStdout: true
                    ).trim()
                    echo "MR 创建结果: ${result}"
                }
            }
        }

        stage('Create Release Tag') {
            when {
                branch 'main'
            }
            steps {
                sh """
                    python3 scripts/gitlab_client.py create-tag \
                        ${PROJECT_ID} \
                        v${env.BUILD_NUMBER} \
                        main \
                        "Release v${env.BUILD_NUMBER}"
                """
            }
        }
    }
}

3.4 直接使用 requests 调用 GitLab REST API

不想引入 python-gitlab 库时，可直接用 requests 调用 REST API，更灵活但需要自行处理分页和错误。

python 复制代码 
# scripts/gitlab_rest.py
import requests
import os
import json

class GitLabREST:
    def __init__(self):
        self.base_url = os.environ.get('GITLAB_URL', 'https://gitlab.com')
        self.token = os.environ.get('GITLAB_TOKEN')
        self.headers = {'PRIVATE-TOKEN': self.token}

    def _get(self, path, params=None):
        url = f"{self.base_url}/api/v4{path}"
        resp = requests.get(url, headers=self.headers, params=params, timeout=30)
        resp.raise_for_status()
        return resp.json()

    def _post(self, path, data=None):
        url = f"{self.base_url}/api/v4{path}"
        resp = requests.post(url, headers=self.headers, json=data, timeout=30)
        resp.raise_for_status()
        return resp.json()

    def _delete(self, path):
        url = f"{self.base_url}/api/v4{path}"
        resp = requests.delete(url, headers=self.headers, timeout=30)
        resp.raise_for_status()
        return resp.status_code == 204

    def get_pipeline_status(self, project_id, pipeline_id):
        return self._get(f'/projects/{project_id}/pipelines/{pipeline_id}')

    def retry_pipeline(self, project_id, pipeline_id):
        return self._post(f'/projects/{project_id}/pipelines/{pipeline_id}/retry')

    def get_merge_request_diff(self, project_id, mr_iid):
        changes = self._get(f'/projects/{project_id}/merge_requests/{mr_iid}/changes')
        return {
            'changed_files': [f['new_path'] for f in changes.get('changes', [])]
        }

    def search_code(self, project_id, search_text, ref='main'):
        return self._get(f'/projects/{project_id}/search', params={
            'scope': 'blobs', 'search': search_text, 'ref': ref
        })

if __name__ == '__main__':
    client = GitLabREST()
    project_id = os.environ.get('PROJECT_ID')
    if project_id:
        status = client.get_pipeline_status(project_id, os.environ.get('PIPELINE_ID', '1'))
        print(json.dumps(status, indent=2, ensure_ascii=False))

3.5 GitLab Webhook 事件处理

在 Pipeline 中处理 GitLab Webhook 事件，实现自动化流程（如 MR 审查、自动合并）。

python 复制代码 
# scripts/handle_webhook.py
import json
import sys

def handle_merge_request_event(event_data):
    action = event_data.get('object_attributes', {}).get('action')
    source_branch = event_data.get('object_attributes', {}).get('source_branch', '')
    target_branch = event_data.get('object_attributes', {}).get('target_branch', '')

    result = {
        'action': action,
        'source_branch': source_branch,
        'target_branch': target_branch,
        'should_auto_merge': False,
        'should_notify': True
    }

    if action == 'open' and target_branch == 'develop' and source_branch.startswith('feature/'):
        result['should_auto_merge'] = True
        result['auto_merge_reason'] = 'feature → develop 自动合并'

    return result

def handle_push_event(event_data):
    ref = event_data.get('ref', '')
    branch = ref.replace('refs/heads/', '')
    commits = event_data.get('commits', [])

    changed_files = []
    for commit in commits:
        changed_files.extend(commit.get('added', []))
        changed_files.extend(commit.get('modified', []))
        changed_files.extend(commit.get('removed', []))

    return {
        'branch': branch,
        'commit_count': len(commits),
        'changed_files': list(set(changed_files)),
        'should_trigger_deploy': branch in ['main', 'develop'],
        'should_run_tests': True
    }

if __name__ == '__main__':
    event_file = sys.argv[1]
    with open(event_file, 'r') as f:
        event = json.load(f)

    if 'merge_request' in event.get('object_kind', ''):
        result = handle_merge_request_event(event)
    elif event.get('object_kind') == 'push':
        result = handle_push_event(event)
    else:
        result = {'action': 'unknown', 'kind': event.get('object_kind')}

    print(json.dumps(result, indent=2, ensure_ascii=False))

四、发送通知

4.1 钉钉/飞书/企业微信通知

构建结果推送到即时通讯工具，是 CI/CD 中最常见的通知需求。

钉钉机器人通知
python 复制代码 
# scripts/notify_dingtalk.py
import requests
import hmac
import hashlib
import base64
import urllib.parse
import time
import os

class DingTalkNotifier:
    def __init__(self, webhook, secret=None):
        self.webhook = webhook
        self.secret = secret

    def _sign_url(self):
        if not self.secret:
            return self.webhook
        timestamp = str(round(time.time() * 1000))
        string_to_sign = f'{timestamp}\n{self.secret}'
        hmac_code = hmac.new(
            self.secret.encode('utf-8'),
            string_to_sign.encode('utf-8'),
            digestmod=hashlib.sha256
        ).digest()
        sign = urllib.parse.quote_plus(base64.b64encode(hmac_code))
        return f"{self.webhook}&timestamp={timestamp}&sign={sign}"

    def send_markdown(self, title, text):
        url = self._sign_url()
        payload = {
            "msgtype": "markdown",
            "markdown": {"title": title, "text": text}
        }
        resp = requests.post(url, json=payload, timeout=10)
        return resp.json()

    def send_build_result(self, job_name, build_number, result, url, duration=''):
        status_emoji = '✅' if result == 'SUCCESS' else '❌'
        text = f"""### {status_emoji} 构建通知

- **任务**: {job_name}
- **构建号**: #{build_number}
- **结果**: {result}
- **耗时**: {duration}
- **详情**: [查看构建]({url})
"""
        return self.send_markdown(f"{status_emoji} {job_name} #{build_number}", text)

if __name__ == '__main__':
    notifier = DingTalkNotifier(
        webhook=os.environ.get('DINGTALK_WEBHOOK'),
        secret=os.environ.get('DINGTALK_SECRET')
    )
    notifier.send_build_result(
        job_name=os.environ.get('JOB_NAME', 'unknown'),
        build_number=os.environ.get('BUILD_NUMBER', '0'),
        result=os.environ.get('BUILD_RESULT', 'SUCCESS'),
        url=os.environ.get('BUILD_URL', ''),
        duration=os.environ.get('BUILD_DURATION', '')
    )
飞书机器人通知
python 复制代码 
# scripts/notify_feishu.py
import requests
import os

class FeishuNotifier:
    def __init__(self, webhook):
        self.webhook = webhook

    def send_build_result(self, job_name, build_number, result, url):
        color = 'green' if result == 'SUCCESS' else 'red'
        emoji = '✅' if result == 'SUCCESS' else '❌'
        payload = {
            "msg_type": "interactive",
            "card": {
                "header": {
                    "title": {"tag": "plain_text", "content": f"{emoji} {job_name} #{build_number}"},
                    "template": color
                },
                "elements": [{
                    "tag": "div",
                    "text": {
                        "tag": "lark_md",
                        "content": f"**任务**: {job_name}\n**构建号**: #{build_number}\n**结果**: {result}\n[查看详情]({url})"
                    }
                }]
            }
        }
        resp = requests.post(self.webhook, json=payload, timeout=10)
        return resp.json()

if __name__ == '__main__':
    notifier = FeishuNotifier(os.environ.get('FEISHU_WEBHOOK'))
    notifier.send_build_result(
        job_name=os.environ.get('JOB_NAME', 'unknown'),
        build_number=os.environ.get('BUILD_NUMBER', '0'),
        result=os.environ.get('BUILD_RESULT', 'SUCCESS'),
        url=os.environ.get('BUILD_URL', '')
    )

4.2 Email 通知（HTML 报告）

发送格式化的 HTML 邮件通知，包含构建详情和测试报告链接。

python 复制代码 
# scripts/notify_email.py
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
import os

def send_build_email(smtp_host, smtp_port, sender, password, recipients,
                     job_name, build_number, result, build_url):
    msg = MIMEMultipart('alternative')
    msg['Subject'] = f"[Jenkins] {job_name} #{build_number} - {result}"
    msg['From'] = sender
    msg['To'] = ', '.join(recipients)

    status_color = '#28a745' if result == 'SUCCESS' else '#dc3545'
    status_text = '构建成功' if result == 'SUCCESS' else '构建失败'

    html = f"""
    <html><body>
    <div style="max-width:600px;margin:0 auto;font-family:sans-serif;">
        <div style="background:{status_color};color:white;padding:20px;text-align:center;">
            <h2>{status_text}: {job_name} #{build_number}</h2>
        </div>
        <table style="width:100%;border-collapse:collapse;">
            <tr><td style="padding:8px;border:1px solid #ddd;">任务</td>
                <td style="padding:8px;border:1px solid #ddd;">{job_name}</td></tr>
            <tr><td style="padding:8px;border:1px solid #ddd;">构建号</td>
                <td style="padding:8px;border:1px solid #ddd;">#{build_number}</td></tr>
            <tr><td style="padding:8px;border:1px solid #ddd;">结果</td>
                <td style="padding:8px;border:1px solid #ddd;color:{status_color};">{result}</td></tr>
        </table>
        <div style="text-align:center;margin-top:20px;">
            <a href="{build_url}" style="background:#0366d6;color:white;padding:10px 20px;
               text-decoration:none;border-radius:5px;">查看构建详情</a>
        </div>
    </div>
    </body></html>
    """

    msg.attach(MIMEText(html, 'html', 'utf-8'))

    with smtplib.SMTP(smtp_host, smtp_port) as server:
        server.starttls()
        server.login(sender, password)
        server.sendmail(sender, recipients, msg.as_string())

if __name__ == '__main__':
    send_build_email(
        smtp_host=os.environ.get('SMTP_HOST', 'smtp.example.com'),
        smtp_port=int(os.environ.get('SMTP_PORT', '587')),
        sender=os.environ.get('SMTP_SENDER'),
        password=os.environ.get('SMTP_PASSWORD'),
        recipients=os.environ.get('EMAIL_RECIPIENTS', '').split(','),
        job_name=os.environ.get('JOB_NAME', 'unknown'),
        build_number=os.environ.get('BUILD_NUMBER', '0'),
        result=os.environ.get('BUILD_RESULT', 'SUCCESS'),
        build_url=os.environ.get('BUILD_URL', ''),
    )

4.3 Pipeline 中统一通知

groovy 复制代码 
pipeline {
    agent any
    environment {
        DINGTALK_WEBHOOK = credentials('dingtalk-webhook')
        DINGTALK_SECRET = credentials('dingtalk-secret')
    }
    stages {
        stage('Build') {
            steps {
                sh 'make build'
            }
        }
    }
    post {
        success {
            sh '''
                export BUILD_RESULT=SUCCESS
                export BUILD_DURATION="${currentBuild.durationString}"
                python3 scripts/notify_dingtalk.py
            '''
        }
        failure {
            sh '''
                export BUILD_RESULT=FAILURE
                export BUILD_DURATION="${currentBuild.durationString}"
                python3 scripts/notify_dingtalk.py
            '''
        }
    }
}

五、生成报告

5.1 测试报告汇总

将多个测试框架的输出汇总为统一格式的报告，便于 Jenkins 展示和归档。

python 复制代码 
# scripts/test_report.py
import xml.etree.ElementTree as ET
import json
import os
import sys
from datetime import datetime

def parse_junit_xml(xml_file):
    tree = ET.parse(xml_file)
    root = tree.getroot()

    suites = []
    for testsuite in root.iter('testsuite'):
        suite = {
            'name': testsuite.get('name'),
            'tests': int(testsuite.get('tests', 0)),
            'failures': int(testsuite.get('failures', 0)),
            'errors': int(testsuite.get('errors', 0)),
            'skipped': int(testsuite.get('skipped', 0)),
            'time': float(testsuite.get('time', 0)),
            'testcases': []
        }

        for testcase in testsuite.iter('testcase'):
            tc = {
                'name': testcase.get('name'),
                'classname': testcase.get('classname'),
                'time': float(testcase.get('time', 0)),
                'status': 'passed'
            }
            if testcase.find('failure') is not None:
                tc['status'] = 'failed'
            elif testcase.find('error') is not None:
                tc['status'] = 'error'
            elif testcase.find('skipped') is not None:
                tc['status'] = 'skipped'
            suite['testcases'].append(tc)

        suites.append(suite)

    return suites

def generate_summary_report(xml_files, output_file='test-summary.json'):
    all_suites = []
    total = {'tests': 0, 'passed': 0, 'failed': 0, 'errors': 0, 'skipped': 0, 'time': 0}

    for xml_file in xml_files:
        suites = parse_junit_xml(xml_file)
        for suite in suites:
            total['tests'] += suite['tests']
            total['failed'] += suite['failures']
            total['errors'] += suite['errors']
            total['skipped'] += suite['skipped']
            total['time'] += suite['time']
            total['passed'] += suite['tests'] - suite['failures'] - suite['errors'] - suite['skipped']
        all_suites.extend(suites)

    report = {
        'timestamp': datetime.now().isoformat(),
        'summary': total,
        'suites': all_suites,
        'pass_rate': f"{(total['passed'] / total['tests'] * 100):.1f}%" if total['tests'] > 0 else '0%'
    }

    with open(output_file, 'w', encoding='utf-8') as f:
        json.dump(report, f, indent=2, ensure_ascii=False)

    print(f"测试报告已生成: {output_file}")
    print(f"总计: {total['tests']} 个测试, 通过率: {report['pass_rate']}")

    if total['failed'] > 0 or total['errors'] > 0:
        sys.exit(1)

if __name__ == '__main__':
    xml_files = sys.argv[1:]
    generate_summary_report(xml_files)

5.2 变更日志自动生成

基于 Git 提交历史自动生成 Changelog，用于版本发布。

python 复制代码 
# scripts/generate_changelog.py
import subprocess
import os
import sys
from datetime import datetime

def get_git_log(from_tag=None, to_ref='HEAD'):
    cmd = ['git', 'log', '--pretty=format:%H|%s|%an|%ad', '--date=short']
    if from_tag:
        cmd.append(f'{from_tag}..{to_ref}')
    else:
        cmd.extend(['-50', to_ref])

    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.stdout.strip().split('\n')

def categorize_commits(log_lines):
    categories = {
        'feat': [], 'fix': [], 'perf': [],
        'refactor': [], 'docs': [], 'chore': [], 'other': []
    }

    for line in log_lines:
        if not line.strip():
            continue
        parts = line.split('|')
        if len(parts) < 4:
            continue

        subject, author, date = parts[1], parts[2], parts[3]
        entry = f"- {subject} ({author}, {date})"

        lower = subject.lower()
        if lower.startswith('feat'):
            categories['feat'].append(entry)
        elif lower.startswith('fix'):
            categories['fix'].append(entry)
        elif lower.startswith('perf'):
            categories['perf'].append(entry)
        elif lower.startswith('refactor'):
            categories['refactor'].append(entry)
        elif lower.startswith('docs'):
            categories['docs'].append(entry)
        elif lower.startswith('chore'):
            categories['chore'].append(entry)
        else:
            categories['other'].append(entry)

    return categories

def generate_changelog(version, from_tag=None):
    log_lines = get_git_log(from_tag)
    categories = categorize_commits(log_lines)

    today = datetime.now().strftime('%Y-%m-%d')
    lines = [f"## {version} ({today})\n"]

    label_map = {
        'feat': '🚀 新功能', 'fix': '🐛 修复', 'perf': '⚡ 性能优化',
        'refactor': '♻️ 重构', 'docs': '📝 文档', 'chore': '🔧 其他',
        'other': '📋 其他变更'
    }

    for key, label in label_map.items():
        if categories[key]:
            lines.append(f"\n### {label}\n")
            lines.extend(categories[key])

    changelog = '\n'.join(lines)

    with open('CHANGELOG.md', 'w', encoding='utf-8') as f:
        f.write(changelog)

    print(f"Changelog 已生成: {version}")

if __name__ == '__main__':
    version = sys.argv[1] if len(sys.argv) > 1 else f"v{os.environ.get('BUILD_NUMBER', '0')}"
    from_tag = sys.argv[2] if len(sys.argv) > 2 else None
    generate_changelog(version, from_tag)
groovy 复制代码 
stage('Generate Changelog') {
    steps {
        sh "python3 scripts/generate_changelog.py v${env.BUILD_NUMBER} vPrevious"
        archiveArtifacts artifacts: 'CHANGELOG.md'
    }
}

六、配置管理与密钥处理

6.1 动态生成部署配置

根据环境变量和模板，动态生成不同环境的部署配置。

python 复制代码 
# scripts/generate_deploy_config.py
import yaml
import os

ENVIRONMENTS = {
    'dev': {
        'replicas': 1, 'cpu': '100m', 'memory': '128Mi',
        'db_host': 'dev-mysql.internal', 'debug': True, 'log_level': 'DEBUG'
    },
    'staging': {
        'replicas': 2, 'cpu': '500m', 'memory': '512Mi',
        'db_host': 'staging-mysql.internal', 'debug': False, 'log_level': 'INFO'
    },
    'production': {
        'replicas': 4, 'cpu': '1000m', 'memory': '1Gi',
        'db_host': 'prod-mysql.internal', 'debug': False, 'log_level': 'WARNING'
    }
}

def generate_k8s_config(env, app_name, image_tag):
    config = ENVIRONMENTS.get(env, ENVIRONMENTS['dev'])

    deployment = {
        'apiVersion': 'apps/v1',
        'kind': 'Deployment',
        'metadata': {'name': app_name, 'labels': {'app': app_name, 'env': env}},
        'spec': {
            'replicas': config['replicas'],
            'selector': {'matchLabels': {'app': app_name}},
            'template': {
                'metadata': {'labels': {'app': app_name, 'env': env}},
                'spec': {
                    'containers': [{
                        'name': app_name,
                        'image': f"{app_name}:{image_tag}",
                        'resources': {
                            'requests': {'cpu': config['cpu'], 'memory': config['memory']},
                            'limits': {'cpu': config['cpu'], 'memory': config['memory']}
                        },
                        'env': [
                            {'name': 'APP_ENV', 'value': env},
                            {'name': 'DB_HOST', 'value': config['db_host']},
                            {'name': 'DEBUG', 'value': str(config['debug'])},
                            {'name': 'LOG_LEVEL', 'value': config['log_level']}
                        ]
                    }]
                }
            }
        }
    }

    output_dir = f'k8s/output/{env}'
    os.makedirs(output_dir, exist_ok=True)
    with open(f'{output_dir}/deployment.yaml', 'w') as f:
        yaml.dump(deployment, f, default_flow_style=False, allow_unicode=True)

    print(f"部署配置已生成: {output_dir}/deployment.yaml")

if __name__ == '__main__':
    generate_deploy_config(
        env=os.environ.get('DEPLOY_ENV', 'dev'),
        app_name=os.environ.get('APP_NAME', 'myapp'),
        image_tag=os.environ.get('APP_VERSION', 'latest')
    )

6.2 密钥与敏感信息检查

在 Pipeline 中检查代码和配置中是否泄露了敏感信息。

python 复制代码 
# scripts/secret_manager.py
import os
import sys
import re
import json

def validate_no_secrets_leaked(file_path):
    patterns = {
        'AWS Key': r'AKIA[0-9A-Z]{16}',
        'Private Key': r'-----BEGIN (?:RSA |EC )?PRIVATE KEY-----',
        'Generic Secret': r'(?:password|secret|token|api_key)\s*[=:]\s*["\']?[^\s"\']{8,}',
    }

    with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
        content = f.read()

    leaks = []
    for pattern_name, pattern in patterns.items():
        matches = re.findall(pattern, content, re.IGNORECASE)
        if matches:
            leaks.append({'pattern': pattern_name, 'matches': len(matches)})

    return leaks

def mask_secrets_in_text(text, secrets):
    for secret_name, secret_value in secrets.items():
        if secret_value and len(str(secret_value)) > 3:
            masked = str(secret_value)[:2] + '*' * (len(str(secret_value)) - 4) + str(secret_value)[-2:]
            text = text.replace(str(secret_value), masked)
    return text

if __name__ == '__main__':
    action = sys.argv[1]

    if action == 'check-leaks':
        target_file = sys.argv[2]
        leaks = validate_no_secrets_leaked(target_file)
        if leaks:
            print(f"⚠️ 发现敏感信息泄露: {json.dumps(leaks, indent=2)}")
            sys.exit(1)
        else:
            print("✅ 未发现敏感信息泄露")
groovy 复制代码 
stage('Check Secrets') {
    steps {
        sh 'python3 scripts/secret_manager.py check-leaks config/production.env'
    }
}

七、代码质量与安全扫描

7.1 代码质量门禁

在 Pipeline 中执行代码质量检查，不达标则阻断流水线。

python 复制代码 
# scripts/quality_gate.py
import subprocess
import sys

class QualityGate:
    def __init__(self):
        self.checks = []

    def add_check(self, name, command, threshold, operator='>='):
        self.checks.append({
            'name': name, 'command': command,
            'threshold': threshold, 'operator': operator
        })

    def run(self):
        all_passed = True
        for check in self.checks:
            try:
                result = subprocess.run(
                    check['command'], shell=True,
                    capture_output=True, text=True, timeout=300
                )
                value = float(result.stdout.strip())
            except (ValueError, subprocess.TimeoutExpired):
                value = 0

            if check['operator'] == '>=':
                passed = value >= check['threshold']
            elif check['operator'] == '<=':
                passed = value <= check['threshold']
            else:
                passed = value == check['threshold']

            status = '✅' if passed else '❌'
            print(f"{status} {check['name']}: {value} (阈值: {check['operator']}{check['threshold']})")

            if not passed:
                all_passed = False

        return all_passed

if __name__ == '__main__':
    gate = QualityGate()

    gate.add_check('测试覆盖率', 'python3 -c "import json; d=json.load(open(\'coverage.json\')); print(d[\'totals\'][\'percent_covered\'])"', 80)
    gate.add_check('Lint 错误数', 'cat lint-result.txt | grep -c "error" || echo 0', 0, operator='<=')
    gate.add_check('安全漏洞数', 'cat safety-result.txt | wc -l', 0, operator='<=')

    if not gate.run():
        print("\n❌ 质量门禁未通过，阻断流水线")
        sys.exit(1)
    else:
        print("\n✅ 质量门禁通过")

7.2 依赖安全检查

检查 Python 依赖是否存在已知安全漏洞。

python 复制代码 
# scripts/security_check.py
import subprocess
import json
import sys

def check_safety():
    result = subprocess.run(
        ['pip', 'audit', '--format', 'json'],
        capture_output=True, text=True
    )
    if result.stdout:
        try:
            vulnerabilities = json.loads(result.stdout)
            critical = [v for v in vulnerabilities if v.get('severity') == 'critical']

            print(f"安全扫描结果: {len(vulnerabilities)} 个漏洞, 严重: {len(critical)}")

            if critical:
                print("❌ 发现严重漏洞，阻断部署")
                for v in critical:
                    print(f"  - {v.get('name')} {v.get('version')}")
                return False
        except json.JSONDecodeError:
            pass

    print("✅ 安全扫描通过")
    return True

def check_requirements_pin():
    unpinned = []
    with open('requirements.txt', 'r') as f:
        for line_num, line in enumerate(f, 1):
            line = line.strip()
            if line and not line.startswith('#') and '==' not in line:
                unpinned.append({'line': line_num, 'package': line})

    if unpinned:
        print(f"⚠️ 发现 {len(unpinned)} 个未固定版本的依赖")
        return False

    print("✅ 所有依赖版本已固定")
    return True

if __name__ == '__main__':
    action = sys.argv[1] if len(sys.argv) > 1 else 'all'

    passed = True
    if action in ('all', 'safety'):
        if not check_safety():
            passed = False
    if action in ('all', 'pin'):
        if not check_requirements_pin():
            passed = False

    if not passed:
        sys.exit(1)

八、与其他系统集成

8.1 Jira REST API 速查表

以下表格汇总了 Pipeline 中最常用的 Jira REST API。Jira Cloud 和 Jira Server/Data Center 的 API 路径略有差异，以下以 Jira Cloud 为准。

认证方式 ：使用 Basic Auth（邮箱 + API Token 编码为 Base64），Header 携带 Authorization: Basic <base64(email:token)>

基础路径 ：{JIRA_URL}/rest/api/2（v2）或 /rest/api/3（v3）

8.1.1 Issue 操作
功能 方法 路径 请求参数 返回值（关键字段）
获取Issue GET /issue/{issueKey} fields: 返回字段列表(逗号分隔); expand: changelog/renderedFields key, fields.summary, fields.status, fields.assignee, fields.created
创建Issue POST /issue Body: fields: {project.key, summary , description, issuetype.name*, assignee, labels, priority} key, self, id
更新Issue PUT /issue/{issueKey} Body: fields: {需更新的字段} 204 No Content（成功）
删除Issue DELETE /issue/{issueKey} deleteSubtasks: bool 204 No Content
批量获取Issue POST /issue/bulkfetch Body: issueKeys: key列表, fields: 字段列表 issues: Issue列表
8.1.2 Issue 状态流转
功能 方法 路径 请求参数 返回值（关键字段）
获取可用流转 GET /issue/{issueKey}/transitions 无 transitions: {`id`, `name`, `to`: {`id`, `name`}, `hasScreen`}
执行状态流转 POST /issue/{issueKey}/transitions Body: transition: {id*}, fields: {流转所需字段} 204 No Content
8.1.3 Issue 评论
功能 方法 路径 请求参数 返回值（关键字段）
获取评论列表 GET /issue/{issueKey}/comment startAt, maxResults, orderBy comments: {`id`, `body`, `author`, `created`, `updated`}
添加评论 POST /issue/{issueKey}/comment Body: body*: 评论内容(支持Wiki标记); visibility id, body, author, created
更新评论 PUT /issue/{issueKey}/comment/{commentId} Body: body*: 新评论内容 id, body, updated
删除评论 DELETE /issue/{issueKey}/comment/{commentId} 无 204 No Content
8.1.4 搜索（JQL）
功能 方法 路径 请求参数 返回值（关键字段）
JQL搜索Issue GET /search jql*: JQL查询语句; startAt; maxResults; fields: 返回字段 total, maxResults, issues: {`key`, `fields`}
JQL搜索Issue(POST) POST /search Body: jql*, startAt, maxResults, fields, validateQuery 同上（适合复杂JQL）
8.1.5 项目
功能 方法 路径 请求参数 返回值（关键字段）
获取项目 GET /project/{projectKeyOrId} 无 key, name, projectTypeKey, lead, projectCategory
列出项目 GET /project recent: 最近访问数量 项目列表，每项含 key, name, projectTypeKey
获取项目状态 GET /project/{projectKeyOrId}/statuses 无 状态列表，每项含 name, id, statuses: {`name`, `id`}
8.1.6 用户
功能 方法 路径 请求参数 返回值（关键字段）
搜索用户 GET /user/search query*: 搜索关键词; maxResults 用户列表，每项含 accountId, displayName, emailAddress
获取当前用户 GET /myself 无 accountId, displayName, emailAddress, active

8.2 Jira API Python 封装

基于 Jira API 速查表，封装常用的 Jira 操作，用于 Pipeline 中自动关联 Issue 和构建。

python 复制代码 
# scripts/jira_client.py
import requests
import os
import base64
import json
import re
import sys

class JiraClient:
    def __init__(self):
        self.base_url = os.environ.get('JIRA_URL', 'https://jira.example.com')
        email = os.environ.get('JIRA_EMAIL')
        token = os.environ.get('JIRA_TOKEN')
        auth = base64.b64encode(f"{email}:{token}".encode()).decode()
        self.headers = {
            'Authorization': f'Basic {auth}',
            'Content-Type': 'application/json'
        }

    def _get(self, path, params=None):
        resp = requests.get(
            f"{self.base_url}/rest/api/2{path}",
            headers=self.headers, params=params, timeout=30
        )
        resp.raise_for_status()
        return resp.json()

    def _post(self, path, data=None):
        resp = requests.post(
            f"{self.base_url}/rest/api/2{path}",
            headers=self.headers, json=data, timeout=30
        )
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def _put(self, path, data=None):
        resp = requests.put(
            f"{self.base_url}/rest/api/2{path}",
            headers=self.headers, json=data, timeout=30
        )
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def get_issue(self, issue_key, fields=None):
        params = {}
        if fields:
            params['fields'] = ','.join(fields)
        return self._get(f'/issue/{issue_key}', params)

    def create_issue(self, project_key, summary, description='', issue_type='Task', **extra_fields):
        fields = {
            'project': {'key': project_key},
            'summary': summary,
            'description': description,
            'issuetype': {'name': issue_type}
        }
        fields.update(extra_fields)
        result = self._post('/issue', {'fields': fields})
        return result['key'] if result else None

    def update_issue(self, issue_key, **fields):
        self._put(f'/issue/{issue_key}', {'fields': fields})

    def transition_issue(self, issue_key, transition_name):
        transitions = self._get(f'/issue/{issue_key}/transitions')
        for t in transitions.get('transitions', []):
            if t['name'].lower() == transition_name.lower():
                self._post(f'/issue/{issue_key}/transitions', {
                    'transition': {'id': t['id']}
                })
                return True
        return False

    def add_comment(self, issue_key, body):
        result = self._post(f'/issue/{issue_key}/comment', {'body': body})
        return result is not None

    def search_issues(self, jql, max_results=50, fields=None):
        params = {'jql': jql, 'maxResults': max_results}
        if fields:
            params['fields'] = ','.join(fields)
        result = self._get('/search', params)
        return result.get('issues', [])

    def get_project(self, project_key):
        return self._get(f'/project/{project_key}')

    def extract_issue_keys_from_branch(self, branch_name):
        matches = re.findall(r'[A-Z][A-Z0-9]+-\d+', branch_name)
        return matches

    def link_build_to_issues(self, branch_name, build_url, build_result):
        issue_keys = self.extract_issue_keys_from_branch(branch_name)
        for key in issue_keys:
            self.add_comment(key,
                f"🏗 Jenkins 构建: [{build_result}] {build_url}\n"
                f"分支: {branch_name}"
            )
        return issue_keys

if __name__ == '__main__':
    client = JiraClient()
    action = sys.argv[1]

    if action == 'create':
        key = client.create_issue(
            project_key=sys.argv[2],
            summary=sys.argv[3],
            description=sys.argv[4] if len(sys.argv) > 4 else ''
        )
        print(json.dumps({'key': key}))

    elif action == 'transition':
        result = client.transition_issue(sys.argv[2], sys.argv[3])
        print(json.dumps({'success': result}))

    elif action == 'comment':
        result = client.add_comment(sys.argv[2], sys.argv[3])
        print(json.dumps({'success': result}))

    elif action == 'search':
        issues = client.search_issues(sys.argv[2])
        print(json.dumps([{'key': i['key'], 'summary': i['fields']['summary']} for i in issues],
                         indent=2, ensure_ascii=False))

    elif action == 'link-build':
        keys = client.link_build_to_issues(
            branch_name=os.environ.get('BRANCH_NAME', ''),
            build_url=os.environ.get('BUILD_URL', ''),
            build_result=os.environ.get('BUILD_RESULT', 'SUCCESS')
        )
        print(json.dumps({'linked_issues': keys}))

8.3 调用 SonarQube API

获取 SonarQube 扫描结果，实现质量门禁自动化。

python 复制代码 
# scripts/sonarqube_client.py
import requests
import os
import json
import sys

class SonarQubeClient:
    def __init__(self):
        self.base_url = os.environ.get('SONAR_URL', 'http://sonarqube:9000')
        self.token = os.environ.get('SONAR_TOKEN')
        self.auth = (self.token, '')

    def get_quality_gate_status(self, project_key):
        resp = requests.get(
            f"{self.base_url}/api/qualitygates/project_status",
            params={'projectKey': project_key},
            auth=self.auth
        )
        return resp.json().get('projectStatus', {}).get('status', 'UNKNOWN')

    def get_measures(self, project_key, metrics):
        resp = requests.get(
            f"{self.base_url}/api/measures/component",
            params={'component': project_key, 'metricKeys': ','.join(metrics)},
            auth=self.auth
        )
        measures = {}
        for m in resp.json().get('component', {}).get('measures', []):
            measures[m['metric']] = m.get('value', 'N/A')
        return measures

if __name__ == '__main__':
    client = SonarQubeClient()
    project_key = os.environ.get('SONAR_PROJECT_KEY', 'my-project')

    status = client.get_quality_gate_status(project_key)
    measures = client.get_measures(project_key, [
        'coverage', 'bugs', 'vulnerabilities', 'code_smells',
        'duplicated_lines_density', 'security_hotspots'
    ])

    print(f"质量门禁状态: {status}")
    print(f"指标: {json.dumps(measures, indent=2)}")

    if status != 'OK':
        print("❌ SonarQube 质量门禁未通过")
        sys.exit(1)

8.4 Jenkins REST API 速查表

以下表格汇总了 Pipeline 中常用的 Jenkins REST API，可用于跨 Job 触发构建、获取构建结果、管理节点等场景。

认证方式 ：使用 Basic Auth（用户名 + API Token），Header 携带 Authorization: Basic <base64(user:token)>

基础路径 ：{JENKINS_URL}

CRUMB 防护 ：POST 请求需先获取 Crumb，Header 携带 Jenkins-Crumb: <crumb_value>

8.4.1 Job 操作
功能 方法 路径 请求参数 返回值（关键字段）
获取Job信息 GET /job/{job_name}/api/json tree: 过滤字段; depth: 嵌套深度 name, url, color, lastBuild, lastSuccessfulBuild, builds
获取Job配置 GET /job/{job_name}/config.xml 无 XML 格式的 Job 配置
更新Job配置 POST /job/{job_name}/config.xml Body: XML 配置内容 200 OK
触发构建(无参) POST /job/{job_name}/build 无 201 Created，Location Header 指向队列项
触发构建(有参) POST /job/{job_name}/buildWithParameters param1=value1&param2=value2(Query或Form) 201 Created，Location Header 指向队列项
禁用Job POST /job/{job_name}/disable 无 200 OK
启用Job POST /job/{job_name}/enable 无 200 OK
删除Job POST /job/{job_name}/doDelete 无 302 重定向
获取文件夹下Job列表 GET /job/{folder_name}/api/json tree: jobsname,url,color jobs: {`name`, `url`, `color`}
8.4.2 Build 操作
功能 方法 路径 请求参数 返回值（关键字段）
获取构建信息 GET /job/{job_name}/{build_number}/api/json tree: 过滤字段 number, result, url, duration, timestamp, actions, artifacts
获取构建日志 GET /job/{job_name}/{build_number}/consoleText 无 纯文本日志内容
获取构建日志(渐进) GET /job/{job_name}/{build_number}/logText/progressiveText start: 字节偏移量 文本内容 + X-Text-Size Header
获取构建产物列表 GET /job/{job_name}/{build_number}/api/json?tree=artifacts[*] 无 artifacts: {`fileName`, `relativePath`}
下载构建产物 GET /job/{job_name}/{build_number}/artifact/{relative_path} relative_path: 产物相对路径 文件二进制内容
停止构建 POST /job/{job_name}/{build_number}/stop 无 200 OK
获取测试结果 GET /job/{job_name}/{build_number}/testReport/api/json 无 suites, passCount, failCount, skipCount
8.4.3 队列操作
功能 方法 路径 请求参数 返回值（关键字段）
获取队列 GET /queue/api/json 无 items: {`id`, `url`, `why`, `task`: {`name`, `url`}, `stuck`}
获取队列项详情 GET /queue/item/{queue_id}/api/json 无 id, task, why, executable(构建开始后含number, url), cancelled
取消队列项 POST /queue/item/{queue_id}/cancelQueue 无 200 OK
8.4.4 系统与节点
功能 方法 路径 请求参数 返回值（关键字段）
获取系统信息 GET /api/json tree: 过滤字段 nodeName, nodeDescription, numExecutors, jobs
获取Crumb GET /crumbIssuer/api/json 无 crumb, crumbRequestField
安静模式(暂停构建) POST /quietDown 无 200 OK
取消安静模式 POST /cancelQuietDown 无 200 OK
获取节点列表 GET /computer/api/json 无 computer: {`displayName`, `offline`, `idle`, `numExecutors`}
获取节点详情 GET /computer/{node_name}/api/json 无 displayName, offline, idle, monitorData, executors
安全重启 POST /safeRestart 无 200 OK
8.4.5 凭证与视图
功能 方法 路径 请求参数 返回值（关键字段）
获取视图列表 GET /api/json?tree=views[*] 无 views: {`name`, `url`, `jobs`}
获取视图内Job GET /view/{view_name}/api/json tree: jobsname,url,color jobs: {`name`, `url`, `color`}
获取凭证列表 GET /credentials/store/system/domain/_/api/json 无 credentials: {`id`, `typeName`, `description`}

8.5 Jenkins API Python 封装

基于 Jenkins API 速查表，封装常用的 Jenkins 远程操作，用于跨 Job 触发、构建结果查询、节点管理等场景。

python 复制代码 
# scripts/jenkins_client.py
import requests
import os
import json
import sys
import time
import base64

class JenkinsClient:
    def __init__(self):
        self.base_url = os.environ.get('JENKINS_URL', 'http://localhost:8080')
        user = os.environ.get('JENKINS_USER', 'admin')
        token = os.environ.get('JENKINS_TOKEN')
        auth = base64.b64encode(f"{user}:{token}".encode()).decode()
        self.headers = {
            'Authorization': f'Basic {auth}',
            'Content-Type': 'application/json'
        }
        self._crumb = None
        self._crumb_field = None

    def _get_crumb(self):
        if self._crumb is None:
            resp = requests.get(
                f"{self.base_url}/crumbIssuer/api/json",
                headers=self.headers, timeout=10
            )
            if resp.status_code == 200:
                data = resp.json()
                self._crumb = data['crumb']
                self._crumb_field = data['crumbRequestField']

    def _get(self, path, params=None):
        resp = requests.get(
            f"{self.base_url}{path}",
            headers=self.headers, params=params, timeout=30
        )
        resp.raise_for_status()
        return resp.json()

    def _post(self, path, data=None):
        self._get_crumb()
        headers = dict(self.headers)
        if self._crumb:
            headers[self._crumb_field] = self._crumb
        resp = requests.post(
            f"{self.base_url}{path}",
            headers=headers, data=data, timeout=30
        )
        resp.raise_for_status()
        return resp

    def get_job_info(self, job_name):
        return self._get(f'/job/{job_name}/api/json', {'tree': 'name,url,color,lastBuild[number,url,result],builds[number,url,result]'})

    def trigger_build(self, job_name, params=None):
        if params:
            resp = self._post(f'/job/{job_name}/buildWithParameters', params)
        else:
            resp = self._post(f'/job/{job_name}/build')
        queue_url = resp.headers.get('Location', '')
        queue_id = queue_url.rstrip('/').split('/')[-1] if queue_url else None
        return {'queue_id': queue_id, 'queue_url': queue_url}

    def get_build_info(self, job_name, build_number):
        return self._get(f'/job/{job_name}/{build_number}/api/json',
                         {'tree': 'number,result,url,duration,timestamp,actions[parameters],artifacts[fileName,relativePath]'})

    def get_build_log(self, job_name, build_number):
        resp = requests.get(
            f"{self.base_url}/job/{job_name}/{build_number}/consoleText",
            headers=self.headers, timeout=30
        )
        resp.raise_for_status()
        return resp.text

    def get_last_build_result(self, job_name):
        info = self._get(f'/job/{job_name}/api/json', {'tree': 'lastBuild[number,result,building]'})
        if not info.get('lastBuild'):
            return None
        return info['lastBuild']

    def wait_for_build(self, job_name, build_number, interval=10, timeout=600):
        start = time.time()
        while time.time() - start < timeout:
            info = self.get_build_info(job_name, build_number)
            if not info.get('building', True):
                return info
            time.sleep(interval)
        raise TimeoutError(f"构建 {job_name}#{build_number} 超时")

    def trigger_and_wait(self, job_name, params=None, interval=10, timeout=600):
        result = self.trigger_build(job_name, params)
        queue_id = result.get('queue_id')
        if not queue_id:
            raise RuntimeError(f"无法获取队列ID: {result}")

        for _ in range(int(timeout / interval)):
            queue_info = self._get(f'/queue/item/{queue_id}/api/json')
            executable = queue_info.get('executable')
            if executable:
                build_number = executable['number']
                build_info = self.wait_for_build(job_name, build_number, interval, timeout)
                return build_info
            if queue_info.get('cancelled'):
                raise RuntimeError(f"构建已被取消: {job_name}")
            time.sleep(interval)

        raise TimeoutError(f"等待构建启动超时: {job_name}")

    def get_queue(self):
        return self._get('/queue/api/json')

    def get_node_status(self):
        data = self._get('/computer/api/json', {'tree': 'computer[displayName,offline,idle,numExecutors]'})
        return data.get('computer', [])

    def get_test_report(self, job_name, build_number):
        return self._get(f'/job/{job_name}/{build_number}/testReport/api/json')

    def download_artifact(self, job_name, build_number, relative_path, output_file):
        resp = requests.get(
            f"{self.base_url}/job/{job_name}/{build_number}/artifact/{relative_path}",
            headers=self.headers, timeout=60
        )
        resp.raise_for_status()
        with open(output_file, 'wb') as f:
            f.write(resp.content)
        return output_file

if __name__ == '__main__':
    client = JenkinsClient()
    action = sys.argv[1]

    if action == 'trigger':
        result = client.trigger_build(sys.argv[2])
        print(json.dumps(result))

    elif action == 'trigger-with-params':
        params = dict(arg.split('=') for arg in sys.argv[3:])
        result = client.trigger_build(sys.argv[2], params)
        print(json.dumps(result))

    elif action == 'build-info':
        info = client.get_build_info(sys.argv[2], int(sys.argv[3]))
        print(json.dumps(info, indent=2))

    elif action == 'last-result':
        result = client.get_last_build_result(sys.argv[2])
        print(json.dumps(result, indent=2))

    elif action == 'build-log':
        log = client.get_build_log(sys.argv[2], int(sys.argv[3]))
        print(log[-5000:] if len(log) > 5000 else log)

    elif action == 'trigger-and-wait':
        result = client.trigger_and_wait(sys.argv[2])
        print(json.dumps({'number': result['number'], 'result': result['result']}, indent=2))

    elif action == 'nodes':
        nodes = client.get_node_status()
        print(json.dumps(nodes, indent=2))

    elif action == 'test-report':
        report = client.get_test_report(sys.argv[2], int(sys.argv[3]))
        print(json.dumps({
            'passCount': report.get('passCount'),
            'failCount': report.get('failCount'),
            'skipCount': report.get('skipCount')
        }, indent=2))

Pipeline 中使用 Jenkins API 示例：

groovy 复制代码 
pipeline {
    agent any
    environment {
        JENKINS_URL = 'http://jenkins:8080'
        JENKINS_USER = 'admin'
        JENKINS_TOKEN = credentials('jenkins-api-token')
    }
    stages {
        stage('Trigger Downstream Job') {
            steps {
                script {
                    def result = sh(
                        script: '''
                            python3 scripts/jenkins_client.py trigger-with-params \
                                deploy-production \
                                ENV=staging \
                                VERSION=${APP_VERSION}
                        ''',
                        returnStdout: true
                    ).trim()
                    echo "触发结果: ${result}"
                }
            }
        }

        stage('Check Upstream Build') {
            steps {
                script {
                    def lastResult = sh(
                        script: 'python3 scripts/jenkins_client.py last-result my-upstream-job',
                        returnStdout: true
                    ).trim()
                    def buildInfo = readJSON(text: lastResult)
                    if (buildInfo.result != 'SUCCESS') {
                        error "上游构建未成功: ${buildInfo.result}"
                    }
                }
            }
        }

        stage('Monitor Nodes') {
            steps {
                sh 'python3 scripts/jenkins_client.py nodes'
            }
        }
    }
}

九、版本管理与发布

9.1 语义化版本自动计算

根据提交信息自动计算下一版本号（遵循 SemVer 规范）。

python 复制代码 
# scripts/version_manager.py
import subprocess
import os
import sys

def get_latest_tag():
    result = subprocess.run(
        ['git', 'describe', '--tags', '--abbrev=0'],
        capture_output=True, text=True
    )
    return result.stdout.strip() if result.returncode == 0 else None

def parse_version(tag):
    parts = tag.lstrip('v').split('.')
    return {
        'major': int(parts[0]) if len(parts) > 0 else 0,
        'minor': int(parts[1]) if len(parts) > 1 else 0,
        'patch': int(parts[2]) if len(parts) > 2 else 0
    }

def get_commit_messages_since_tag(tag=None):
    cmd = ['git', 'log', '--pretty=format:%s']
    if tag:
        cmd.append(f'{tag}..HEAD')
    else:
        cmd.extend(['-50'])
    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.stdout.strip().split('\n')

def calculate_next_version(tag=None, bump_type=None):
    current = parse_version(tag) if tag else {'major': 0, 'minor': 0, 'patch': 0}

    if bump_type:
        if bump_type == 'major':
            current['major'] += 1; current['minor'] = 0; current['patch'] = 0
        elif bump_type == 'minor':
            current['minor'] += 1; current['patch'] = 0
        elif bump_type == 'patch':
            current['patch'] += 1
    else:
        messages = get_commit_messages_since_tag(tag)
        has_breaking = any('BREAKING CHANGE' in m or '!' in m.split(':')[0] for m in messages if m)
        has_feat = any(m.lower().startswith('feat') for m in messages if m)

        if has_breaking:
            current['major'] += 1; current['minor'] = 0; current['patch'] = 0
        elif has_feat:
            current['minor'] += 1; current['patch'] = 0
        else:
            current['patch'] += 1

    return f"{current['major']}.{current['minor']}.{current['patch']}"

if __name__ == '__main__':
    action = sys.argv[1] if len(sys.argv) > 1 else 'next'

    if action == 'next':
        latest = get_latest_tag()
        print(calculate_next_version(latest))
    elif action == 'current':
        print(get_latest_tag() or '0.0.0')
    elif action == 'bump':
        latest = get_latest_tag()
        bump = sys.argv[2] if len(sys.argv) > 2 else None
        print(calculate_next_version(latest, bump))
groovy 复制代码 
stage('Calculate Version') {
    steps {
        script {
            env.NEXT_VERSION = sh(
                script: 'python3 scripts/version_manager.py next',
                returnStdout: true
            ).trim()
            echo "下一版本: ${env.NEXT_VERSION}"
        }
    }
}

十、数据库操作

10.1 数据库迁移前检查与备份

在执行数据库迁移前，检查数据库连接、备份关键数据、验证迁移脚本。

python 复制代码 
# scripts/db_ops.py
import pymysql
import os
import sys
import json
import csv
from datetime import datetime

class DBOperator:
    def __init__(self, host, port, user, password, database):
        self.connection = pymysql.connect(
            host=host, port=int(port), user=user,
            password=password, database=database, charset='utf8mb4'
        )

    def check_connection(self):
        try:
            with self.connection.cursor() as cursor:
                cursor.execute('SELECT 1')
            return {'status': 'ok', 'message': '数据库连接正常'}
        except Exception as e:
            return {'status': 'error', 'message': str(e)}

    def get_table_row_counts(self):
        with self.connection.cursor() as cursor:
            cursor.execute("""
                SELECT table_name, table_rows
                FROM information_schema.tables
                WHERE table_schema = DATABASE()
                ORDER BY table_rows DESC
            """)
            return [{'table': row[0], 'rows': row[1]} for row in cursor.fetchall()]

    def backup_table(self, table_name, backup_dir='backups'):
        os.makedirs(backup_dir, exist_ok=True)
        timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
        backup_file = f"{backup_dir}/{table_name}_{timestamp}.csv"

        with self.connection.cursor() as cursor:
            cursor.execute(f"SELECT * FROM {table_name}")
            rows = cursor.fetchall()
            columns = [desc[0] for desc in cursor.description]

        with open(backup_file, 'w', newline='', encoding='utf-8') as f:
            writer = csv.writer(f)
            writer.writerow(columns)
            writer.writerows(rows)

        return {'table': table_name, 'rows': len(rows), 'file': backup_file}

    def run_migration_check(self, migration_file):
        with open(migration_file, 'r') as f:
            sql_content = f.read()

        dangerous_patterns = [
            ('DROP TABLE', '删除表'),
            ('DROP DATABASE', '删除数据库'),
            ('TRUNCATE', '清空表'),
        ]

        warnings = []
        for pattern, desc in dangerous_patterns:
            if pattern.lower() in sql_content.lower():
                warnings.append(f"⚠️ {desc}: 发现 {pattern} 语句")

        return {'file': migration_file, 'warnings': warnings, 'safe': len(warnings) == 0}

    def close(self):
        self.connection.close()

if __name__ == '__main__':
    db = DBOperator(
        host=os.environ.get('DB_HOST', 'localhost'),
        port=os.environ.get('DB_PORT', '3306'),
        user=os.environ.get('DB_USER', 'root'),
        password=os.environ.get('DB_PASSWORD', ''),
        database=os.environ.get('DB_NAME', 'myapp')
    )

    action = sys.argv[1]

    if action == 'check':
        result = db.check_connection()
        print(json.dumps(result, ensure_ascii=False))
        if result['status'] != 'ok':
            sys.exit(1)

    elif action == 'counts':
        counts = db.get_table_row_counts()
        print(json.dumps(counts, indent=2, ensure_ascii=False))

    elif action == 'backup':
        result = db.backup_table(sys.argv[2])
        print(json.dumps(result, ensure_ascii=False))

    elif action == 'migration-check':
        result = db.run_migration_check(sys.argv[2])
        print(json.dumps(result, indent=2, ensure_ascii=False))
        if not result['safe']:
            sys.exit(1)

    db.close()
groovy 复制代码 
stage('DB Pre-Migration Check') {
    steps {
        withCredentials([usernamePassword(credentialsId: 'db-creds',
            usernameVariable: 'DB_USER', passwordVariable: 'DB_PASSWORD')]) {
            sh '''
                python3 scripts/db_ops.py check
                python3 scripts/db_ops.py migration-check alembic/versions/latest.py
                python3 scripts/db_ops.py backup users
            '''
        }
    }
}

十一、完整 Pipeline 示例

将上述所有场景整合到一个完整的 Jenkinsfile 中。

groovy 复制代码 
pipeline {
    agent {
        docker {
            image 'python:3.11-slim'
            args '-v $HOME/.cache/pip:/root/.cache/pip'
        }
    }

    environment {
        APP_NAME = 'myapp'
        GITLAB_TOKEN = credentials('gitlab-api-token')
        DINGTALK_WEBHOOK = credentials('dingtalk-webhook')
        DINGTALK_SECRET = credentials('dingtalk-secret')
        SONAR_TOKEN = credentials('sonar-token')
        DB_CREDS = credentials('db-credentials')
    }

    stages {
        stage('Setup') {
            steps {
                sh '''
                    pip install -r scripts/requirements.txt
                    python3 scripts/version_manager.py current
                '''
            }
        }

        stage('Calculate Version') {
            steps {
                script {
                    env.APP_VERSION = sh(
                        script: 'python3 scripts/version_manager.py next',
                        returnStdout: true
                    ).trim()
                }
            }
        }

        stage('Security Check') {
            steps {
                sh 'python3 scripts/security_check.py all'
            }
        }

        stage('Build') {
            steps {
                sh 'make build'
            }
        }

        stage('Test') {
            steps {
                sh 'make test'
            }
            post {
                always {
                    sh 'python3 scripts/test_report.py reports/*.xml'
                }
            }
        }

        stage('Quality Gate') {
            steps {
                sh 'python3 scripts/quality_gate.py'
            }
        }

        stage('Generate Config') {
            steps {
                sh '''
                    export DEPLOY_ENV=staging
                    python3 scripts/generate_deploy_config.py
                '''
            }
        }

        stage('Generate Changelog') {
            steps {
                sh "python3 scripts/generate_changelog.py v${env.APP_VERSION}"
            }
        }

        stage('Create GitLab Tag') {
            when { branch 'main' }
            steps {
                sh """
                    python3 scripts/gitlab_client.py create-tag \
                        \${PROJECT_ID} \
                        v${env.APP_VERSION} \
                        main \
                        "Release v${env.APP_VERSION}"
                """
            }
        }

        stage('DB Migration Check') {
            steps {
                sh '''
                    export DB_USER=\${DB_CREDS_USR}
                    export DB_PASSWORD=\${DB_CREDS_PSW}
                    python3 scripts/db_ops.py check
                    python3 scripts/db_ops.py migration-check alembic/versions/latest.py
                '''
            }
        }
    }

    post {
        always {
            archiveArtifacts artifacts: 'test-summary.json, CHANGELOG.md', allowEmptyArchive: true
        }
        success {
            sh '''
                export BUILD_RESULT=SUCCESS
                export BUILD_DURATION="${currentBuild.durationString}"
                python3 scripts/notify_dingtalk.py
            '''
        }
        failure {
            sh '''
                export BUILD_RESULT=FAILURE
                export BUILD_DURATION="${currentBuild.durationString}"
                python3 scripts/notify_dingtalk.py
            '''
        }
    }
}

十二、最佳实践与注意事项

12.1 脚本组织规范

复制代码 
project/
├── scripts/                    # Pipeline 使用的 Python 脚本
│   ├── requirements.txt        # Python 依赖
│   ├── gitlab_client.py        # GitLab API 封装
│   ├── notify_dingtalk.py      # 钉钉通知
│   ├── notify_feishu.py        # 飞书通知
│   ├── notify_email.py         # 邮件通知
│   ├── test_report.py          # 测试报告
│   ├── quality_gate.py         # 质量门禁
│   ├── security_check.py       # 安全检查
│   ├── config_processor.py     # 配置处理
│   ├── generate_deploy_config.py  # 部署配置生成
│   ├── generate_changelog.py   # Changelog 生成
│   ├── version_manager.py      # 版本管理
│   ├── db_ops.py               # 数据库操作
│   ├── secret_manager.py       # 密钥管理
│   ├── jira_client.py          # Jira 集成
│   ├── jenkins_client.py       # Jenkins API 集成
│   ├── sonarqube_client.py     # SonarQube 集成
│   └── log_analyzer.py         # 日志分析
├── templates/                  # 配置模板
│   ├── config.yaml.j2
│   └── deployment.yaml.j2
├── Jenkinsfile                 # Pipeline 定义
└── ...

12.2 注意事项

问题 说明 解决方案
环境差异 不同 Jenkins 节点 Python 版本/库不一致 使用 Docker Agent 或 venv
依赖安装慢 每次构建重新安装 pip 包 缓存 pip 包到共享卷
密钥泄露 Token/密码暴露在日志中 使用 credentials() + 环境变量 + 日志脱敏
脚本失败静默 Python 异常被 Groovy 吞掉 使用 set -e + sys.exit(1)
中文编码 中文输出乱码 统一 UTF-8，PYTHONIOENCODING=utf-8
超时 长时间运行的脚本卡住 设置 timeout(time: 10, unit: 'MINUTES')
并发冲突 多构建同时写同一文件 使用 BUILD_NUMBER 隔离工作目录

12.3 性能优化

groovy 复制代码 
// 缓存 pip 依赖
pipeline {
    agent {
        docker {
            image 'python:3.11-slim'
            args '-v $HOME/.cache/pip:/root/.cache/pip'
        }
    }
    stages {
        stage('Install') {
            steps {
                sh 'pip install --cache-dir /root/.cache/pip -r scripts/requirements.txt'
            }
        }
    }
}

// 设置超时
options {
    timeout(time: 30, unit: 'MINUTES')
}

// 设置编码
environment {
    PYTHONIOENCODING = 'utf-8'
    PYTHONUNBUFFERED = '1'
}

12.4 调试技巧

groovy 复制代码 
// 调试：打印环境变量
sh 'python3 -c "import os; [print(f\"{k}={v}\") for k, v in sorted(os.environ.items())]"'

// 调试：打印 Python 版本和路径
sh 'which python3 && python3 --version'

// 调试：捕获 Python 异常
sh '''
    python3 -c "
import traceback
try:
    import my_module
except Exception as e:
    traceback.print_exc()
    import sys; sys.exit(1)
    "
'''

十三、场景速查表

场景 Python 能做什么 关键库
文件处理 解析/修改 YAML/JSON/ENV 配置 pyyaml, jinja2
GitLab API MR/Tag/Branch/文件/Pipeline/Job 操作 python-gitlab, requests
Jira API Issue 创建/状态流转/评论/JQL搜索 requests
Jenkins API 触发构建/获取结果/节点管理/产物下载 requests
通知推送 钉钉/飞书/企业微信/邮件 requests, smtplib
报告生成 测试汇总/质量报告/Changelog xml.etree, json, subprocess
配置管理 多环境配置生成/模板渲染 jinja2, pyyaml
安全检查 密钥泄露检测/依赖漏洞扫描 re, pip-audit
质量门禁 覆盖率/Lint/漏洞阈值检查 subprocess
版本管理 SemVer 自动计算/Tag 创建 subprocess, gitlab
数据库操作 连接检查/数据备份/迁移验证 pymysql
SonarQube 质量门禁/指标获取 requests
日志分析 错误提取/测试结果解析 re, json
Webhook 事件分发/自动合并/通知 json
密钥管理 泄露检测/日志脱敏 re
相关推荐
流浪0011 小时前
C++篇：深入理解 C++ 智能指针：从裸指针到 RAII 的蜕变
开发语言·c++
多彩电脑1 小时前
在Kivy中制造可移动控件
python
丘山望岳1 小时前
二叉搜索双壁——map和set
开发语言·数据结构·c++
瑞雪兆丰年兮1 小时前
[从0开始学Java|第十六、十七天]项目阶段（拼图小游戏）
java·开发语言
AI人工智能+电脑小能手1 小时前
【大白话说Java面试题 第85题】【Mysql篇】第15题：MySQL 的事务中，幻读是怎么解决的？
java·开发语言·数据库·mysql·面试
Zy_Yin1231 小时前
拆解如何用anthropic金融agent做投研
人工智能·python·深度学习·金融·github
清水白石0081 小时前
Python 变量的本质：从“盒子思维”到“引用思维”，彻底理解赋值到底发生了什么
java·python·ajax
yaoxin5211231 小时前
423. Java 日期时间 API - DayOfWeek 和 Month 枚举
开发语言·python
燐妤1 小时前
Python工具使用：Pycharm
python·pycharm
热门推荐
01GitHub 镜像站点02【AI】2026 年具身智能模型和世界模型总结03【踩坑记录 | 第一篇】微软商店无法使用时，如何手动安装 OpenAI Codex？附`.msix`文件系统错误解决方法042026 年 AI 编程工具终极横评：Cursor vs Claude Code vs Copilot vs Windsurf05DeepSeek V4 + Claude Code thinking mode 400 错误修复方案06Codex 接入 DeepSeek API 完整配置文档07裂开！ChatGPT 居然开始要手机号验证，附详细解决方法08CC-Switch & Claude 基于 Linux 服务器安装使用指南09几个好用的ip纯净度检测网站10API Key 登录 Codex 也能用插件了，还支持会话删除和导出