DevOps --- Pipeline和Yaml文件

什么是Pipleine
什么是Yaml文件
Yaml文件的构成

什么是Pipleine

在DevOps中pipeline可以按顺序在一台虚拟机上实现一组特定的操作，包括但不限于

执行git命令，如拉取代码，push代码等

执行任意程序

执行python或bash脚本

...

基于以上操作，pipeline通常可以完成以下场景

持续集成：拉取代码，运行代码质量检查工具，运行集成测试，部署代码等

使用第三方控件对代码进行安全扫描(DAST, Image,SAST等)

定时运行E2E测试

...

什么是Yaml文件

和Json，XML类似，YAML（YAML Ain't Markup Language）是一种数据序列化格式。经常被用于配置文件和数据交换格式，特别适用于各种编程语言之间的数据传递.

在微软的DevOps中，yaml通常用来定义一个pipeline

Yaml文件的构成

一个pipeline 由多个stages组成

一个stage由多个jobs组成

每个Job可以在一个agent上执行（一个Job也可以没有agent）

an agent could be an VM or sometimes an VM can have multiple agents

一个job由多个steps组成

steps可以是task或script，是pipeline的最小构建单元

task是一些Azure DevOps预先设定好的功能，可以直接使用. 常用的task包括,

PythonScript@0

PowerShell@2

Docker@1

CopyFiles@2

PublishTestResults@2

...

可以在ADO中搜索

Trigger

This states what changes trigger the pipeline. For the example in the below code, changes in the main branch alone trigger the pipeline run and not from the feature branches.

Pool

When your build or deployment runs, the system begins one or more jobs. An agent is a computing infrastructure with installed agent software that runs one job at a time. For example, your job could run on a Microsoft-hosted Ubuntu agent. The below pipeline yaml uses "ubuntu-latest"

Variables

Variables are a way to get key bits of data into various parts of the pipeline.

The most common use of variables is to define a value that you can then use in your pipeline.

All variables are strings and are mutable. The value of a variable can change from run to run or job to job of your pipeline.

Example

yaml 复制代码

trigger:
  branches:
    include:
    - main
    exclude:
    - feature_branches

pool:
  vmImage: 'ubuntu-latest'

variables:
  - name: tag
    value: '$(Build.BuildNumber)'
  - name: ImageName
    value: "demo Image"
  - name: python.version
    value: '3.8'

schedules:
 - cron: "0 18 1 * *"
   always: true
   displayName: Monthly, 2nd, 2:00 am (utc+8)
   branches:
     include:
     - master

stages:
  - stage: Lint
    displayName: Format and lint code
    jobs:
      - job: Linting
        steps:
        - script: |
            python3 -m pip install black
            python3 -m pip install pylint
          displayName: "Install dependencies"  

        - script: |
            #app is the folder in which the application code resides
            python3 -m black ./app
          displayName: "Apply black code formatting"

        - script: |
            python3 -m pylint ./app --recursive=true --exit-zero
          displayName: "Static code analysis with pylint"
  
  - stage: Test
    displayName: Unit test
    jobs:
      - job: Test
        steps:
        - script: |
            python3 -m pip install -r requirements.txt
            python3 -m pip install pytest-azurepipelines
          displayName: "Install dependencies"
        
        - script: |
            python3 -m pytest -v -s --junitxml=unittest/junit.xml --cov=. --cov-report=term-missing --cov-report=xml
  
  - stage: Build
    displayName: 'Build and Push Docker Image'
    jobs:
      - job: BuildAndPush
        displayName: 'Build and Push Docker Image'
        steps:
       - task: Docker@1
         displayName: 'Build an image'
        inputs:
        containerregistrytype: 'Azure Container Registry'
        azureSubscriptionEndpoint: 'Service connection name'
        azureContainerRegistry: '<<democontainer-DEV.azurecr.io>>'
        command: 'Build an Image'
        dockerFile: '$(System.DefaultWorkingDirectory)/Dockerfile'
        tags: |
          latest
          $(Build.BuildId)
        imageName: '$(ImageName):$(tag)'

       - task: Docker@1
         displayName: 'Push image to ACR to TEST'
         inputs:
           containerregistrytype: 'Azure Container Registry'
           azureSubscriptionEndpoint: '<<demoserviceconenction>>'
           azureContainerRegistry: 'container registary name-DEV.azurecr.io'
           command: 'Push an image'
           imageName: '$(ImageName):$(tag)'
      
       - task: Docker@1
         displayName: 'Build an image'
         inputs:
           containerregistrytype: 'Azure Container Registry'
           azureSubscriptionEndpoint: 'Service connection name'
           azureContainerRegistry: 'container registary name-PROD.azurecr.io'
           command: 'Build an Image'
           dockerFile: '$(System.DefaultWorkingDirectory)/Dockerfile'
           tags: |
               latest
               $(Build.BuildId)
           imageName: '$(ImageName):$(tag)'
      
       - task: Docker@1
         displayName: 'Push image to ACR to PROD'
         inputs:
           containerregistrytype: 'Azure Container Registry'
           azureSubscriptionEndpoint: 'Service connection name'
           azureContainerRegistry: 'container registary name-PROD.azurecr.io'
           command: 'Push an image'
           imageName: '$(ImageName):$(tag)'      
           
       - task: CopyFiles@2
         inputs:
           SourceFolder: 'kubernetes'
           Contents: '*.yaml'
           TargetFolder: '$(Build.ArtifactStagingDirectory)'
      
       - task: PublishBuildArtifacts@1
         inputs:
           PathtoPublish: '$(Build.ArtifactStagingDirectory)'
           ArtifactName: 'manifest'
           publishLocation: 'Container'