11.1 分布式构建概述
什么是分布式构建
分布式构建定义：
Jenkins分布式构建是指将构建任务分散到多个节点（Agent）上执行的架构模式。
这种模式可以提高构建效率、资源利用率和系统可扩展性。
核心概念：
- Controller（控制器）：Jenkins主节点，负责调度和管理
- Agent（代理）：执行构建任务的工作节点
- Executor（执行器）：Agent上的并发执行单元
- Label（标签）：用于标识和选择特定类型的Agent
- Node（节点）：Controller和Agent的统称分布式构建优势：
性能优势：
- 并行执行：多个构建任务同时运行
- 负载分散：避免单点性能瓶颈
- 资源隔离：不同项目使用独立资源
- 弹性扩展：根据需求动态增减节点
管理优势：
- 环境隔离：不同环境的构建互不影响
- 专用资源：特定任务使用专门配置的节点
- 故障隔离：单个节点故障不影响整体
- 成本优化：按需使用云资源架构模式：
经典架构：
┌─────────────────┐
│ Controller │ ← 主节点（调度、管理、UI）
│ (Master) │
└─────────┬───────┘
│
┌─────┴─────┐
│ │
┌───▼───┐ ┌───▼───┐
│Agent 1│ │Agent 2│ ← 工作节点（执行构建）
└───────┘ └───────┘
现代架构：
┌─────────────────┐
│ Controller │ ← 无状态控制器
└─────────┬───────┘
│
┌─────┴─────┐
│ │
┌───▼───┐ ┌───▼───┐
│ Pod 1 │ │ Pod 2 │ ← Kubernetes Pod
└───────┘ └───────┘节点类型和特性
Controller节点：
职责：
- 用户界面服务
- 构建调度和分发
- 插件管理
- 系统配置管理
- 构建历史存储
- 安全认证和授权
特性：
- 通常不执行构建任务
- 需要持久化存储
- 高可用性要求
- 网络连通性要求Agent节点：
类型分类：
1. 永久Agent（Permanent Agent）
- 长期运行的物理机或虚拟机
- 稳定的网络连接
- 预配置的构建环境
- 适合频繁构建的项目
2. 云Agent（Cloud Agent）
- 按需创建和销毁
- 弹性扩展能力
- 成本优化
- 适合间歇性构建
3. 容器Agent（Container Agent）
- 基于Docker或Kubernetes
- 快速启动和清理
- 环境一致性
- 资源隔离
4. 静态Agent（Static Agent）
- 手动配置和管理
- 固定资源分配
- 简单可靠
- 适合小规模环境11.2 Agent配置与管理
添加永久Agent
通过SSH连接：
配置步骤：
1. 准备Agent机器
- 安装Java运行环境
- 配置SSH服务
- 创建Jenkins用户
- 设置工作目录权限
2. 在Jenkins中添加节点
- 管理Jenkins -> 节点管理 -> 新建节点
- 节点名称：agent-linux-01
- 类型：永久代理
3. 节点配置
- 远程工作目录：/home/jenkins/workspace
- 标签：linux java maven
- 用法：尽可能使用这个节点
- 启动方式：通过SSH启动代理
- 主机：192.168.1.100
- 凭据：jenkins-ssh-key
- 主机密钥验证策略：已知主机文件验证策略SSH Agent配置示例：
bash
# 1. 在Agent机器上创建Jenkins用户
sudo useradd -m -s /bin/bash jenkins
sudo mkdir -p /home/jenkins/.ssh
sudo mkdir -p /home/jenkins/workspace
# 2. 配置SSH密钥认证
# 在Controller上生成密钥对
ssh-keygen -t rsa -b 4096 -f jenkins-agent-key
# 3. 将公钥复制到Agent
sudo cp jenkins-agent-key.pub /home/jenkins/.ssh/authorized_keys
sudo chown -R jenkins:jenkins /home/jenkins
sudo chmod 700 /home/jenkins/.ssh
sudo chmod 600 /home/jenkins/.ssh/authorized_keys
# 4. 测试SSH连接
ssh -i jenkins-agent-key jenkins@192.168.1.100通过JNLP连接：
配置步骤：
1. 创建JNLP节点
- 启动方式：通过Java Web Start启动代理
- 或：让Jenkins控制这个Windows从节点
2. 下载agent.jar
wget http://jenkins-server:8080/jnlpJars/agent.jar
3. 启动Agent
java -jar agent.jar -jnlpUrl http://jenkins-server:8080/computer/agent-name/slave-agent.jnlp -secret <secret-key>
4. 创建启动脚本
#!/bin/bash
JENKINS_URL="http://jenkins-server:8080"
AGENT_NAME="agent-linux-02"
SECRET="your-secret-here"
java -jar agent.jar \
-jnlpUrl "${JENKINS_URL}/computer/${AGENT_NAME}/slave-agent.jnlp" \
-secret "${SECRET}" \
-workDir "/home/jenkins/workspace"Windows Agent配置
Windows服务方式：
powershell
# 1. 下载并安装Jenkins Agent
$jenkinsUrl = "http://jenkins-server:8080"
$agentName = "windows-agent-01"
$secret = "your-secret-here"
# 2. 下载agent.jar
Invoke-WebRequest -Uri "$jenkinsUrl/jnlpJars/agent.jar" -OutFile "agent.jar"
# 3. 创建启动脚本
@"
java -jar agent.jar ^
-jnlpUrl "$jenkinsUrl/computer/$agentName/slave-agent.jnlp" ^
-secret "$secret" ^
-workDir "C:\Jenkins\workspace"
"@ | Out-File -FilePath "start-agent.bat" -Encoding ASCII
# 4. 安装为Windows服务
# 使用WinSW或NSSM工具
nssm install JenkinsAgent "C:\Program Files\Java\jdk-11\bin\java.exe"
nssm set JenkinsAgent Parameters "-jar C:\Jenkins\agent.jar -jnlpUrl $jenkinsUrl/computer/$agentName/slave-agent.jnlp -secret $secret"
nssm set JenkinsAgent AppDirectory "C:\Jenkins"
nssm start JenkinsAgentPowerShell DSC配置：
powershell
Configuration JenkinsAgent {
param(
[string]$JenkinsUrl,
[string]$AgentName,
[string]$Secret
)
Import-DscResource -ModuleName PSDesiredStateConfiguration
Node localhost {
# 创建Jenkins目录
File JenkinsDirectory {
DestinationPath = "C:\Jenkins"
Type = "Directory"
Ensure = "Present"
}
# 下载agent.jar
Script DownloadAgent {
SetScript = {
Invoke-WebRequest -Uri "$using:JenkinsUrl/jnlpJars/agent.jar" -OutFile "C:\Jenkins\agent.jar"
}
TestScript = {
Test-Path "C:\Jenkins\agent.jar"
}
GetScript = {
@{ Result = (Test-Path "C:\Jenkins\agent.jar") }
}
DependsOn = "[File]JenkinsDirectory"
}
# 创建启动脚本
File StartScript {
DestinationPath = "C:\Jenkins\start-agent.bat"
Contents = @"
java -jar C:\Jenkins\agent.jar ^
-jnlpUrl "$JenkinsUrl/computer/$AgentName/slave-agent.jnlp" ^
-secret "$Secret" ^
-workDir "C:\Jenkins\workspace"
"@
Type = "File"
Ensure = "Present"
DependsOn = "[Script]DownloadAgent"
}
}
}
# 应用配置
JenkinsAgent -JenkinsUrl "http://jenkins-server:8080" -AgentName "windows-agent-01" -Secret "your-secret"
Start-DscConfiguration -Path .\JenkinsAgent -Wait -Verbose标签和节点选择
标签策略：
标签分类：
1. 操作系统标签
- linux, windows, macos
- ubuntu, centos, rhel
- windows-2019, windows-2022
2. 架构标签
- x86_64, arm64, aarch64
- amd64, i386
3. 软件环境标签
- java-8, java-11, java-17
- maven, gradle, npm
- docker, kubernetes
- python-3.8, node-16
4. 硬件资源标签
- high-memory, high-cpu
- gpu-enabled
- ssd-storage
5. 功能特性标签
- build, test, deploy
- security-scan
- performance-test
6. 环境标签
- dev, staging, prod
- internal, external
- trusted, untrustedPipeline中的节点选择：
groovy
pipeline {
agent none
stages {
stage('Build') {
agent {
label 'linux && java-11 && maven'
}
steps {
sh 'mvn clean compile'
}
}
stage('Test') {
parallel {
stage('Unit Tests') {
agent {
label 'linux && java-11'
}
steps {
sh 'mvn test'
}
}
stage('Integration Tests') {
agent {
label 'linux && docker'
}
steps {
sh 'docker-compose up -d'
sh 'mvn verify -P integration-tests'
sh 'docker-compose down'
}
}
stage('Performance Tests') {
agent {
label 'high-memory && performance-test'
}
steps {
sh 'jmeter -n -t performance-test.jmx'
}
}
}
}
stage('Security Scan') {
agent {
label 'security-scan && trusted'
}
steps {
sh 'sonar-scanner'
sh 'dependency-check.sh'
}
}
stage('Deploy') {
agent {
label 'deploy && prod'
}
when {
branch 'main'
}
steps {
sh 'kubectl apply -f k8s/'
}
}
}
}动态标签选择：
groovy
pipeline {
agent none
stages {
stage('Dynamic Agent Selection') {
steps {
script {
// 根据条件动态选择Agent
def agentLabel = ''
if (env.BRANCH_NAME == 'main') {
agentLabel = 'prod && deploy'
} else if (env.BRANCH_NAME.startsWith('feature/')) {
agentLabel = 'dev && build'
} else if (env.BRANCH_NAME.startsWith('release/')) {
agentLabel = 'staging && test'
}
// 根据文件变化选择Agent
def changes = currentBuild.changeSets
def hasDockerfile = false
def hasJavaFiles = false
changes.each { changeSet ->
changeSet.each { change ->
change.affectedFiles.each { file ->
if (file.path.contains('Dockerfile')) {
hasDockerfile = true
}
if (file.path.endsWith('.java')) {
hasJavaFiles = true
}
}
}
}
if (hasDockerfile) {
agentLabel += ' && docker'
}
if (hasJavaFiles) {
agentLabel += ' && java-11'
}
echo "选择的Agent标签: ${agentLabel}"
// 使用选择的Agent执行构建
node(agentLabel) {
checkout scm
sh 'echo "在Agent ${NODE_NAME} 上执行构建"'
// 执行构建步骤
}
}
}
}
}
}11.3 云Agent配置
AWS EC2 Agent
EC2插件配置：
插件安装：
- 安装 "Amazon EC2" 插件
- 管理Jenkins -> 系统配置 -> Cloud
配置步骤：
1. 添加新的Cloud
- 类型：Amazon EC2
- 名称：AWS-EC2-Cloud
2. AWS配置
- Access Key ID：使用IAM凭据
- Secret Access Key：使用IAM凭据
- Region：us-west-2
- EC2 Key Pair's Private Key：上传私钥文件
3. AMI配置
- AMI ID：ami-0abcdef1234567890
- Instance Type：t3.medium
- Security group names：jenkins-agents
- Remote user：ec2-user
- AMI Type：unix
- Labels：aws linux docker
- Usage：Use this node as much as possibleEC2 Agent模板：
json
{
"amiType": "unix",
"associatePublicIp": false,
"connectBySSHProcess": false,
"connectUsingPublicIp": false,
"customDeviceMapping": "",
"deleteRootOnTermination": true,
"description": "Jenkins Agent for CI/CD",
"ebsOptimized": false,
"iamInstanceProfile": "jenkins-agent-role",
"idleTerminationMinutes": "30",
"initScript": "#!/bin/bash\nyum update -y\nyum install -y docker git\nservice docker start\nusermod -a -G docker ec2-user",
"instanceCapStr": "10",
"jvmopts": "-Xmx1024m",
"labelString": "aws linux docker maven",
"launchTimeoutStr": "300",
"numExecutors": "2",
"remoteAdmin": "ec2-user",
"remoteFS": "/home/ec2-user/jenkins",
"securityGroups": "jenkins-agents",
"stopOnTerminate": false,
"subnetId": "subnet-12345678",
"tags": [
{
"name": "Name",
"value": "jenkins-agent"
},
{
"name": "Environment",
"value": "ci-cd"
}
],
"tmpDir": "/tmp",
"type": "t3.medium",
"useEphemeralDevices": false,
"userData": ""
}Terraform配置EC2 Agent：
hcl
# variables.tf
variable "jenkins_controller_ip" {
description = "Jenkins Controller IP address"
type = string
}
variable "key_pair_name" {
description = "EC2 Key Pair name"
type = string
}
# main.tf
resource "aws_security_group" "jenkins_agents" {
name_prefix = "jenkins-agents-"
description = "Security group for Jenkins agents"
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["${var.jenkins_controller_ip}/32"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "jenkins-agents"
}
}
resource "aws_iam_role" "jenkins_agent_role" {
name = "jenkins-agent-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "ec2.amazonaws.com"
}
}
]
})
}
resource "aws_iam_role_policy" "jenkins_agent_policy" {
name = "jenkins-agent-policy"
role = aws_iam_role.jenkins_agent_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"s3:GetObject",
"s3:PutObject",
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage"
]
Resource = "*"
}
]
})
}
resource "aws_iam_instance_profile" "jenkins_agent_profile" {
name = "jenkins-agent-profile"
role = aws_iam_role.jenkins_agent_role.name
}
resource "aws_launch_template" "jenkins_agent" {
name_prefix = "jenkins-agent-"
image_id = "ami-0abcdef1234567890" # Amazon Linux 2
instance_type = "t3.medium"
key_name = var.key_pair_name
vpc_security_group_ids = [aws_security_group.jenkins_agents.id]
iam_instance_profile {
name = aws_iam_instance_profile.jenkins_agent_profile.name
}
user_data = base64encode(templatefile("${path.module}/user_data.sh", {
jenkins_controller_ip = var.jenkins_controller_ip
}))
tag_specifications {
resource_type = "instance"
tags = {
Name = "jenkins-agent"
Environment = "ci-cd"
}
}
}用户数据脚本：
bash
#!/bin/bash
# user_data.sh
# 更新系统
yum update -y
# 安装必要软件
yum install -y java-11-openjdk docker git maven
# 启动Docker
systemctl start docker
systemctl enable docker
usermod -a -G docker ec2-user
# 安装Node.js
curl -fsSL https://rpm.nodesource.com/setup_16.x | bash -
yum install -y nodejs
# 创建Jenkins工作目录
mkdir -p /home/ec2-user/jenkins
chown ec2-user:ec2-user /home/ec2-user/jenkins
# 配置SSH
echo "${jenkins_controller_ip} jenkins-controller" >> /etc/hosts
# 安装AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install
# 配置日志
echo "Jenkins Agent initialized at $(date)" >> /var/log/jenkins-agent.logKubernetes Agent
Kubernetes插件配置：
yaml
# jenkins-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: jenkins
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
namespace: jenkinsJenkins Kubernetes配置：
插件安装：
- 安装 "Kubernetes" 插件
- 管理Jenkins -> 系统配置 -> Cloud
配置步骤：
1. 添加Kubernetes Cloud
- 名称：kubernetes
- Kubernetes URL：https://kubernetes.default.svc.cluster.local
- Kubernetes Namespace：jenkins
- Credentials：使用ServiceAccount Token
2. Pod模板配置
- 名称：jenkins-agent
- 命名空间：jenkins
- 标签：kubernetes docker maven
- 用法：Use this node as much as possiblePipeline中使用Kubernetes Agent：
groovy
pipeline {
agent {
kubernetes {
yaml """
apiVersion: v1
kind: Pod
spec:
containers:
- name: maven
image: maven:3.8.1-openjdk-11
command:
- cat
tty: true
volumeMounts:
- name: maven-cache
mountPath: /root/.m2
- name: docker
image: docker:20.10.7-dind
securityContext:
privileged: true
volumeMounts:
- name: docker-sock
mountPath: /var/run/docker.sock
- name: kubectl
image: bitnami/kubectl:latest
command:
- cat
tty: true
volumes:
- name: maven-cache
hostPath:
path: /tmp/maven-cache
- name: docker-sock
hostPath:
path: /var/run/docker.sock
"""
}
}
stages {
stage('Build') {
steps {
container('maven') {
sh 'mvn clean compile'
}
}
}
stage('Test') {
steps {
container('maven') {
sh 'mvn test'
}
}
}
stage('Package') {
steps {
container('maven') {
sh 'mvn package'
}
}
}
stage('Build Image') {
steps {
container('docker') {
sh 'docker build -t myapp:${BUILD_NUMBER} .'
}
}
}
stage('Deploy') {
steps {
container('kubectl') {
sh 'kubectl apply -f k8s/deployment.yaml'
}
}
}
}
}动态Pod模板：
groovy
def createPodTemplate(Map config) {
def podYaml = """
apiVersion: v1
kind: Pod
metadata:
labels:
jenkins: agent
spec:
containers:
"""
// 添加基础容器
podYaml += """
- name: jnlp
image: jenkins/inbound-agent:latest
args: ['\$(JENKINS_SECRET)', '\$(JENKINS_NAME)']
"""
// 根据配置添加容器
if (config.maven) {
podYaml += """
- name: maven
image: maven:3.8.1-openjdk-11
command: ['cat']
tty: true
volumeMounts:
- name: maven-cache
mountPath: /root/.m2
"""
}
if (config.docker) {
podYaml += """
- name: docker
image: docker:20.10.7-dind
securityContext:
privileged: true
volumeMounts:
- name: docker-sock
mountPath: /var/run/docker.sock
"""
}
if (config.node) {
podYaml += """
- name: node
image: node:16-alpine
command: ['cat']
tty: true
"""
}
// 添加卷
podYaml += """
volumes:
"""
if (config.maven) {
podYaml += """
- name: maven-cache
hostPath:
path: /tmp/maven-cache
"""
}
if (config.docker) {
podYaml += """
- name: docker-sock
hostPath:
path: /var/run/docker.sock
"""
}
return podYaml
}
pipeline {
agent {
kubernetes {
yaml createPodTemplate([
maven: true,
docker: true,
node: false
])
}
}
stages {
stage('Build') {
steps {
container('maven') {
sh 'mvn clean package'
}
}
}
stage('Docker Build') {
steps {
container('docker') {
sh 'docker build -t myapp:${BUILD_NUMBER} .'
}
}
}
}
}11.4 负载均衡与调度
构建调度策略
调度算法：
Jenkins调度策略：
1. 标签匹配（Label Matching）
- 精确匹配：agent标签完全匹配job要求
- 子集匹配：agent标签包含job要求的所有标签
- 表达式匹配：支持逻辑运算符（&&, ||, !）
2. 负载均衡（Load Balancing）
- 轮询调度：依次分配到可用节点
- 最少连接：分配到当前负载最轻的节点
- 加权轮询：根据节点性能分配权重
3. 亲和性调度（Affinity Scheduling）
- 节点亲和性：优先使用特定节点
- 任务亲和性：相关任务在同一节点执行
- 反亲和性：避免在同一节点执行冲突任务
4. 资源感知调度（Resource-Aware Scheduling）
- CPU使用率：避免过载节点
- 内存使用率：确保足够内存
- 磁盘空间：检查可用存储空间
- 网络带宽：考虑网络性能自定义调度策略：
groovy
// 在共享库中实现智能调度
def selectOptimalAgent(Map requirements) {
def availableNodes = Jenkins.instance.nodes.findAll { node ->
node.computer.online && !node.computer.temporarilyOffline
}
def suitableNodes = availableNodes.findAll { node ->
def nodeLabels = node.labelString.split(' ') as Set
def requiredLabels = requirements.labels as Set
// 检查标签匹配
if (!requiredLabels.every { nodeLabels.contains(it) }) {
return false
}
// 检查资源要求
def computer = node.computer
def executors = computer.executors
def busyExecutors = executors.findAll { it.busy }
def cpuUsage = (busyExecutors.size() / executors.size()) * 100
if (cpuUsage > requirements.maxCpuUsage ?: 80) {
return false
}
// 检查内存使用
def memoryMonitor = computer.getMonitorData()['hudson.node_monitors.SwapSpaceMonitor']
if (memoryMonitor && memoryMonitor.availablePhysicalMemory < (requirements.minMemoryMB ?: 1024) * 1024 * 1024) {
return false
}
return true
}
if (suitableNodes.empty) {
error "没有找到满足要求的节点: ${requirements}"
}
// 选择负载最轻的节点
def optimalNode = suitableNodes.min { node ->
def computer = node.computer
def executors = computer.executors
def busyExecutors = executors.findAll { it.busy }
return busyExecutors.size() / executors.size()
}
return optimalNode.nodeName
}
// 在Pipeline中使用
pipeline {
agent none
stages {
stage('Smart Scheduling') {
steps {
script {
def requirements = [
labels: ['linux', 'docker', 'high-memory'],
maxCpuUsage: 70,
minMemoryMB: 4096
]
def selectedAgent = selectOptimalAgent(requirements)
echo "选择的Agent: ${selectedAgent}"
node(selectedAgent) {
checkout scm
sh 'echo "在最优Agent上执行构建"'
// 执行构建步骤
}
}
}
}
}
}队列管理
构建队列优化：
groovy
// 队列管理脚本
import jenkins.model.Jenkins
import hudson.model.Queue
import hudson.model.queue.QueueTaskFuture
def optimizeBuildQueue() {
def jenkins = Jenkins.instance
def queue = jenkins.queue
// 获取队列中的任务
def queuedItems = queue.items
echo "当前队列中有 ${queuedItems.length} 个任务"
// 按优先级排序
def prioritizedItems = queuedItems.sort { item ->
def priority = 0
// 主分支构建优先级更高
if (item.task.name.contains('main') || item.task.name.contains('master')) {
priority += 100
}
// 生产部署优先级最高
if (item.task.name.contains('deploy') && item.task.name.contains('prod')) {
priority += 200
}
// 热修复优先级很高
if (item.task.name.contains('hotfix')) {
priority += 150
}
// 测试任务优先级较低
if (item.task.name.contains('test')) {
priority -= 50
}
return -priority // 负数用于降序排列
}
// 重新排列队列（这是一个简化的示例）
prioritizedItems.eachWithIndex { item, index ->
echo "优先级 ${index + 1}: ${item.task.name}"
}
}
// 定期执行队列优化
pipeline {
agent any
triggers {
cron('*/5 * * * *') // 每5分钟执行一次
}
stages {
stage('Queue Optimization') {
steps {
script {
optimizeBuildQueue()
}
}
}
}
}队列监控和告警：
groovy
// 队列监控脚本
def monitorBuildQueue() {
def jenkins = Jenkins.instance
def queue = jenkins.queue
def queuedItems = queue.items
// 检查队列长度
if (queuedItems.length > 20) {
def message = "⚠️ 构建队列过长: ${queuedItems.length} 个任务等待执行"
// 发送告警
slackSend(
channel: '#ops-alerts',
color: 'warning',
message: message
)
// 分析队列中的任务类型
def taskTypes = [:]
queuedItems.each { item ->
def taskName = item.task.name
def taskType = 'other'
if (taskName.contains('build')) taskType = 'build'
else if (taskName.contains('test')) taskType = 'test'
else if (taskName.contains('deploy')) taskType = 'deploy'
taskTypes[taskType] = (taskTypes[taskType] ?: 0) + 1
}
def analysis = taskTypes.collect { type, count ->
"${type}: ${count}"
}.join(', ')
echo "队列任务分析: ${analysis}"
}
// 检查长时间等待的任务
def now = System.currentTimeMillis()
def longWaitingTasks = queuedItems.findAll { item ->
(now - item.inQueueSince) > 30 * 60 * 1000 // 30分钟
}
if (!longWaitingTasks.empty) {
def message = "🕐 发现长时间等待的任务: ${longWaitingTasks.size()} 个"
longWaitingTasks.each { item ->
def waitTime = (now - item.inQueueSince) / (60 * 1000) // 分钟
echo "任务 ${item.task.name} 已等待 ${waitTime.intValue()} 分钟"
}
slackSend(
channel: '#ops-alerts',
color: 'danger',
message: message
)
}
}资源监控
节点资源监控：
groovy
// 节点监控脚本
import jenkins.model.Jenkins
import hudson.node_monitors.*
def monitorNodeResources() {
def jenkins = Jenkins.instance
def nodes = jenkins.nodes + [jenkins] // 包括master节点
nodes.each { node ->
def computer = node.computer
if (!computer.online) {
echo "节点 ${node.nodeName} 离线"
return
}
def monitors = computer.getMonitorData()
// CPU监控
def cpuMonitor = monitors['hudson.node_monitors.ArchitectureMonitor']
// 内存监控
def memoryMonitor = monitors['hudson.node_monitors.SwapSpaceMonitor']
if (memoryMonitor) {
def totalMemory = memoryMonitor.totalPhysicalMemory
def availableMemory = memoryMonitor.availablePhysicalMemory
def memoryUsage = ((totalMemory - availableMemory) / totalMemory) * 100
echo "节点 ${node.nodeName} 内存使用率: ${memoryUsage.round(2)}%"
if (memoryUsage > 90) {
slackSend(
channel: '#ops-alerts',
color: 'danger',
message: "🚨 节点 ${node.nodeName} 内存使用率过高: ${memoryUsage.round(2)}%"
)
}
}
// 磁盘空间监控
def diskMonitor = monitors['hudson.node_monitors.DiskSpaceMonitor']
if (diskMonitor) {
def freeSpace = diskMonitor.size
def totalSpace = diskMonitor.size // 这里需要根据实际API调整
if (freeSpace < 1024 * 1024 * 1024) { // 小于1GB
slackSend(
channel: '#ops-alerts',
color: 'warning',
message: "⚠️ 节点 ${node.nodeName} 磁盘空间不足: ${(freeSpace / (1024*1024*1024)).round(2)}GB"
)
}
}
// 响应时间监控
def responseMonitor = monitors['hudson.node_monitors.ResponseTimeMonitor']
if (responseMonitor && responseMonitor.average > 5000) { // 5秒
slackSend(
channel: '#ops-alerts',
color: 'warning',
message: "🐌 节点 ${node.nodeName} 响应时间过长: ${responseMonitor.average}ms"
)
}
// 执行器状态
def executors = computer.executors
def busyExecutors = executors.findAll { it.busy }
def executorUsage = (busyExecutors.size() / executors.size()) * 100
echo "节点 ${node.nodeName} 执行器使用率: ${executorUsage.round(2)}%"
}
}
// 生成资源报告
def generateResourceReport() {
def jenkins = Jenkins.instance
def nodes = jenkins.nodes + [jenkins]
def report = [
timestamp: new Date().toString(),
nodes: []
]
nodes.each { node ->
def computer = node.computer
def nodeInfo = [
name: node.nodeName,
online: computer.online,
labels: node.labelString,
executors: computer.executors.size(),
busyExecutors: computer.executors.findAll { it.busy }.size()
]
if (computer.online) {
def monitors = computer.getMonitorData()
// 添加监控数据
def memoryMonitor = monitors['hudson.node_monitors.SwapSpaceMonitor']
if (memoryMonitor) {
nodeInfo.memory = [
total: memoryMonitor.totalPhysicalMemory,
available: memoryMonitor.availablePhysicalMemory,
usage: ((memoryMonitor.totalPhysicalMemory - memoryMonitor.availablePhysicalMemory) / memoryMonitor.totalPhysicalMemory) * 100
]
}
def diskMonitor = monitors['hudson.node_monitors.DiskSpaceMonitor']
if (diskMonitor) {
nodeInfo.disk = [
free: diskMonitor.size
]
}
}
report.nodes.add(nodeInfo)
}
// 保存报告
def reportJson = groovy.json.JsonBuilder(report).toPrettyString()
writeFile file: "resource-report-${env.BUILD_NUMBER}.json", text: reportJson
// 归档报告
archiveArtifacts artifacts: "resource-report-${env.BUILD_NUMBER}.json"
return report
}本章小结
本章详细介绍了Jenkins分布式构建的各个方面：
- 分布式构建概述：理解分布式架构的优势和基本概念
- Agent配置与管理：学习各种类型Agent的配置方法
- 云Agent配置：掌握AWS EC2和Kubernetes Agent的使用
- 负载均衡与调度：了解构建调度策略和资源管理
分布式构建是Jenkins的核心特性之一，正确配置和管理分布式环境对于提高构建效率和系统可靠性至关重要。
下一章预告
下一章我们将学习Jenkins的监控与日志管理，包括性能监控、日志分析和故障排除。
练习与思考
理论练习
-
架构设计：
- 设计适合团队的分布式构建架构
- 分析不同Agent类型的适用场景
- 规划节点标签和调度策略
-
资源规划：
- 评估构建资源需求
- 设计弹性扩展策略
- 考虑成本优化方案
实践练习
-
Agent配置：
- 配置SSH和JNLP Agent
- 设置云Agent（AWS或Kubernetes）
- 实现动态Agent调度
-
监控实现：
- 部署资源监控脚本
- 配置告警机制
- 生成资源使用报告
思考题
-
性能优化：
- 如何优化分布式构建的性能？
- 如何处理Agent之间的负载不均衡？
- 如何减少构建等待时间？
-
可靠性保障：
- 如何确保分布式环境的高可用性？
- 如何处理Agent故障和网络问题？
- 如何实现构建任务的故障转移？