Cypress自动化Github workflow

开水好喝2024-10-01 11:59

目录

Workflow

.github/workflows/build.yaml

jobs

build-deploy:

.github/wrokflows/execute-tests.yaml

输入参数

jobs

restart_failed_dp:

seed_data:

execute-tests:

[Docker File](#Docker File)

docker/Dockerfile.base

docker/service-dockerfile.base

Deploy

deploy/test-execute

Workflow

.github/workflows/build.yaml

创建docker image并推送到Amazon ECR

jobs

build-deploy:
bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Configure our environment based on the branch, or if this is a PR build
        if [[ '${{ github.ref }}' == refs/heads/${MAIN_BRANCH} ]]; then
            echo "::set-output name=tag::${{ github.sha }}"
            source .github/auto.env
            cat .github/auto.env >> ${GITHUB_ENV}
        else
          echo "This isn't either the main branch or a pull request. Exiting..."
          exit 1
        fi

        echo "::set-output name=kubernetes-namespace::${KUBERNETES_NAMESPACE}"
    - name: Build, tag, and push image to Amazon ECR
      env:
        IMAGE_TAG: ${{ steps.prep.outputs.tag }}
        GITHUB_TOKEN: ${{ secrets.ORG_TOKEN }}
      run: |
        # This is needed because the Github Action environment variables have issues if the value contains spaces (e.g. 'EXTRA_DOCKER_ARGS')
        source .github/global.env

        # Build and push the image using the SHA tag
        docker build -f ${{ env.DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${EXTRA_DOCKER_ARGS} .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG}

        docker build -f ${{ env.SERVICE_DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest

        # Push this image with the 'latest' tag if this is a "main branch" build
        if [[ '${{ github.ref }}' == 'refs/heads/${{ env.MAIN_BRANCH }}' ]]; then
          docker tag ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
          docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
        fi

备注：

以上脚本执行了下列命令

复制代码 
docker build -f docker/Dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest ${EXTRA_DOCKER_ARGS} .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
复制代码 
docker build -f docker/service-dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest

.github/wrokflows/execute-tests.yaml

输入参数

environment， 必填，选择一个测试环境，可选值为qa，staging等

platform，必填，选择哪一端的代码执行，可选值为web/android/ios等，主要是mobile的seed data也在该workflow中

instanceCount，必填，选择调用多少AWS集群机来执行本次任务

runId，选填，填入需要重跑的test run在testrails上面的id号

reseed_test_data，必填，选择是否执行seed data，销毁并重新生成站点测试数据

push_report，必填，选择是否发送测试报告到99u

image_tag，必填，选择是否获取代码库中的最新代码

jobs

restart_failed_dp:

执行./deploy/at_restart_failed_dp建立DP索引​​​

bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        fi

        chmod 777 ./deploy/at_restart_failed_dp
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Restart Failed DP
      id: at_restart_failed_dp
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        AT_DP_PASSWORD: ${{ secrets.AT_DP_PASSWORD }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
      with:
        entrypoint: ./deploy/at_restart_failed_dp

备注：

  • global.env存储docker注册服务地址、AWS身份认证信息、docker名称、镜像名称、主干分支名、cypress服务端口等全局参数信息
  • auto.env存储AWS Region信息、K8S命名空间、AWS身份认证信息、IMAGE_TAG和自动化测试报告的S3存放url及目录信息
  • qa.env存储被测系统所在的AWS Region信息、AWS身份认证信息、API接口域名信息等环境参数
seed_data:

执行./deploy/at_data_setup调用one-eye的pod造AT所需数据

bash 复制代码 
runs-on: self-hosted
    needs: restart_failed_dp
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "dev" ]]; then
          source .github/dev.env
          cat .github/dev.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "nd" ]]; then
          source .github/nd.env
          cat .github/nd.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        else
          echo "Error env" $ENVIRONMENT
          exit 1
        fi
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Data Setup
      id: at_data_setup
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
        AT_DATA_POD_NAME: ${{ env.AT_DATA_POD_NAME }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
        SEED_DATA_PASSWORD: ${{ env.SEED_DATA_PASSWORD }}
        PLATFORM: ${{ env.PLATFORM }}
      with:
        entrypoint: ./deploy/at_data_setup
execute-tests:

执行./deploy/test-execute启动K8S，分别启动initiator和worker进入测试执行

bash 复制代码 
runs-on: self-hosted
    timeout-minutes: 330
    needs: trigger_airflow
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Source our environment variables for the "auto" account (where Automodo resides)
        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        UNIQUE_ID=$(date +%s)
        echo UNIQUE_ID=$UNIQUE_ID >> $GITHUB_ENV

        IDENTITY_TOKEN=$(cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token)
        echo IDENTITY_TOKEN=${IDENTITY_TOKEN} >> $GITHUB_ENV
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume helm role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: Run Tests
      id: run-tests
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        STAGE: auto
        RELEASE_NAME: ${{ env.APP_NAME }}
        IMAGE_NAME: ${{ env.IMAGE_NAME }}
        IMAGE_TAG: ${{ github.event.inputs.image_tag }}
        HELM_AWS_ROLE_ARN: ${{ env.HELM_AWS_ROLE_ARN }}
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
      with:
        entrypoint: ./deploy/test-execute

Docker File

docker/Dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

# 安装 Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable

ARG ENVIRONMENT=docker

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./
RUN npm ci --no-optional

ENV ENVIRONMENT=
ENV DIRECTOR_ADDRESS=
ENV S3_REPORT_BUCKET_URL=
ENV S3_REPORT_BUCKET=
ENV AWS_REGION=
ENV RUN_ID=
ENV CI_BUILD_ID=
ENV TYPE=

CMD ENVIRONMENT=${ENVIRONMENT} AWS_REGION=${AWS_REGION} S3_REPORT_BUCKET_URL=${S3_REPORT_BUCKET_URL} S3_REPORT_BUCKET=${S3_REPORT_BUCKET} DIRECTOR_ADDRESS=${DIRECTOR_ADDRESS} RUN_ID=${RUN_ID} CI_BUILD_ID=${CI_BUILD_ID} TYPE=${TYPE} node ./cypress/scripts/cicd.js

docker/service-dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./

RUN npm ci --no-optional

CMD node ./cypress/service/service.js

Deploy

deploy/test-execute

bash 复制代码 
#!/bin/bash

# echo ${IDENTITY_TOKEN} > /tmp/token
set -e

unset AWS_PROFILE

export KUBECONFIG="/dev/shm/cicd"
aws --region ${AWS_REGION} eks update-kubeconfig --name=ndpg-${AWS_REGION_SHORT}-${STAGE}-eks-cluster

# kubectl complains if KUBECONFIG is world readable, so lock it down
chmod 600 "${KUBECONFIG}"

cd deploy

if [ "$RESEED_AT_DATA" = "only" ];then
  echo "find flag RESEED_AT_DATA is only, Skip run tests...."
  exit 0
fi

# Clean up the old secrets
kubectl get secrets --no-headers=true | awk '/cypress-automation-config/{print $1}'| xargs  kubectl delete secrets || true

# Deploy the director if none exists
if [ "$(kubectl get service -n ${KUBERNETES_NAMESPACE} ${RELEASE_NAME}-director -o jsonpath='{.metadata.name}')" != "${RELEASE_NAME}-director" ]; then
  echo -e "Starting director、dashboard、api and service...\n"
  helmfile -f helmfile-director.yaml apply --suppress-secrets
fi

# Run the initiator job
if [ ! ${RUN_ID} ]; then
  echo -e "Creating the TestRail run...\n"
else
  echo -e "Rerun tests that are not passed in the existing run, ${RUN_ID}...\n"
fi

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
INSTANCES=1 TYPE=initiator JOB_TIMEOUT=300 helmfile -f helmfile.yaml apply --suppress-secrets
#创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Complete --timeout=5m job/cypress-automation-${ENVIRONMENT}-initiator-${UNIQUE_ID}

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
echo -e "Starting workers...\n"
TYPE=worker JOB_TIMEOUT=9000 helmfile -f helmfile.yaml apply --suppress-secrets
echo -e "See the progress at https://pangaea.testrail.io/index.php?/runs/overview/5"
#创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Ready --timeout=10m -l job-name=cypress-automation-${ENVIRONMENT}-worker-${UNIQUE_ID} pods

echo -e "Done."

备注：

（1）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（2）创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}

（3）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（4）创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}

当Dockerfile.base生成的镜像在容器中运行时，会执行./cypress/scripts/cicd.js开始进入测试

当service-dockerfile.base生成的镜像在容器中运行时，会执行./cypress/service/service.js启动事件监听并在监听到执行完成事件时，向99u发送报告消息

其中，initiator和worker的区别在于，initiator会在testrail中创建新的run或者获取re-run的runId

相关推荐
草莓熊Lotso7 小时前
Linux 文件描述符与重定向实战：从原理到 minishell 实现
android·linux·运维·服务器·数据库·c++·人工智能
历程里程碑7 小时前
Linux22 文件系统
linux·运维·c语言·开发语言·数据结构·c++·算法
七夜zippoe15 小时前
CANN Runtime任务描述序列化与持久化源码深度解码
大数据·运维·服务器·cann
Fcy64817 小时前
Linux下 进程(一)（冯诺依曼体系、操作系统、进程基本概念与基本操作）
linux·运维·服务器·进程
袁袁袁袁满17 小时前
Linux怎么查看最新下载的文件
linux·运维·服务器
代码游侠17 小时前
学习笔记——设备树基础
linux·运维·开发语言·单片机·算法
Harvey90317 小时前
通过 Helm 部署 Nginx 应用的完整标准化步骤
linux·运维·nginx·k8s
珠海西格电力科技18 小时前
微电网能量平衡理论的实现条件在不同场景下有哪些差异？
运维·服务器·网络·人工智能·云计算·智慧城市
释怀不想释怀19 小时前
Linux环境变量
linux·运维·服务器
zzzsde19 小时前
【Linux】进程（4）：进程优先级&&调度队列
linux·运维·服务器
热门推荐
01GitHub 镜像站点02Claude Code + GLM4.7 避坑指南：解决 Unable to connect to Anthropic services03openclaw配置教程（linux+局域网ollama）04UV安装并设置国内源05AI 规范驱动开发“三剑客”深度对比：Spec-Kit、Kiro 与 OpenSpec 实战指南06Linux下V2Ray安装配置指南07OpenClaw Chrome扩展使用教程 - 浏览器中继控制08Claude Code Skills 实用使用手册09openclaw使用nginx反代部署过程 与disconnected (1008): pairing required解决10Vue-skills的中文文档