Cypress自动化Github workflow

开水好喝2024-10-01 11:59

目录

Workflow

.github/workflows/build.yaml

jobs

build-deploy:

.github/wrokflows/execute-tests.yaml

输入参数

jobs

restart_failed_dp:

seed_data:

execute-tests:

[Docker File](#Docker File)

docker/Dockerfile.base

docker/service-dockerfile.base

Deploy

deploy/test-execute

Workflow

.github/workflows/build.yaml

创建docker image并推送到Amazon ECR

jobs

build-deploy:
bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Configure our environment based on the branch, or if this is a PR build
        if [[ '${{ github.ref }}' == refs/heads/${MAIN_BRANCH} ]]; then
            echo "::set-output name=tag::${{ github.sha }}"
            source .github/auto.env
            cat .github/auto.env >> ${GITHUB_ENV}
        else
          echo "This isn't either the main branch or a pull request. Exiting..."
          exit 1
        fi

        echo "::set-output name=kubernetes-namespace::${KUBERNETES_NAMESPACE}"
    - name: Build, tag, and push image to Amazon ECR
      env:
        IMAGE_TAG: ${{ steps.prep.outputs.tag }}
        GITHUB_TOKEN: ${{ secrets.ORG_TOKEN }}
      run: |
        # This is needed because the Github Action environment variables have issues if the value contains spaces (e.g. 'EXTRA_DOCKER_ARGS')
        source .github/global.env

        # Build and push the image using the SHA tag
        docker build -f ${{ env.DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${EXTRA_DOCKER_ARGS} .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG}

        docker build -f ${{ env.SERVICE_DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest

        # Push this image with the 'latest' tag if this is a "main branch" build
        if [[ '${{ github.ref }}' == 'refs/heads/${{ env.MAIN_BRANCH }}' ]]; then
          docker tag ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
          docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
        fi

备注：

以上脚本执行了下列命令

复制代码 
docker build -f docker/Dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest ${EXTRA_DOCKER_ARGS} .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
复制代码 
docker build -f docker/service-dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest

.github/wrokflows/execute-tests.yaml

输入参数

environment， 必填，选择一个测试环境，可选值为qa，staging等

platform，必填，选择哪一端的代码执行，可选值为web/android/ios等，主要是mobile的seed data也在该workflow中

instanceCount，必填，选择调用多少AWS集群机来执行本次任务

runId，选填，填入需要重跑的test run在testrails上面的id号

reseed_test_data，必填，选择是否执行seed data，销毁并重新生成站点测试数据

push_report，必填，选择是否发送测试报告到99u

image_tag，必填，选择是否获取代码库中的最新代码

jobs

restart_failed_dp:

执行./deploy/at_restart_failed_dp建立DP索引​​​

bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        fi

        chmod 777 ./deploy/at_restart_failed_dp
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Restart Failed DP
      id: at_restart_failed_dp
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        AT_DP_PASSWORD: ${{ secrets.AT_DP_PASSWORD }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
      with:
        entrypoint: ./deploy/at_restart_failed_dp

备注：

  • global.env存储docker注册服务地址、AWS身份认证信息、docker名称、镜像名称、主干分支名、cypress服务端口等全局参数信息
  • auto.env存储AWS Region信息、K8S命名空间、AWS身份认证信息、IMAGE_TAG和自动化测试报告的S3存放url及目录信息
  • qa.env存储被测系统所在的AWS Region信息、AWS身份认证信息、API接口域名信息等环境参数
seed_data:

执行./deploy/at_data_setup调用one-eye的pod造AT所需数据

bash 复制代码 
runs-on: self-hosted
    needs: restart_failed_dp
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "dev" ]]; then
          source .github/dev.env
          cat .github/dev.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "nd" ]]; then
          source .github/nd.env
          cat .github/nd.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        else
          echo "Error env" $ENVIRONMENT
          exit 1
        fi
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Data Setup
      id: at_data_setup
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
        AT_DATA_POD_NAME: ${{ env.AT_DATA_POD_NAME }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
        SEED_DATA_PASSWORD: ${{ env.SEED_DATA_PASSWORD }}
        PLATFORM: ${{ env.PLATFORM }}
      with:
        entrypoint: ./deploy/at_data_setup
execute-tests:

执行./deploy/test-execute启动K8S，分别启动initiator和worker进入测试执行

bash 复制代码 
runs-on: self-hosted
    timeout-minutes: 330
    needs: trigger_airflow
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Source our environment variables for the "auto" account (where Automodo resides)
        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        UNIQUE_ID=$(date +%s)
        echo UNIQUE_ID=$UNIQUE_ID >> $GITHUB_ENV

        IDENTITY_TOKEN=$(cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token)
        echo IDENTITY_TOKEN=${IDENTITY_TOKEN} >> $GITHUB_ENV
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume helm role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: Run Tests
      id: run-tests
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        STAGE: auto
        RELEASE_NAME: ${{ env.APP_NAME }}
        IMAGE_NAME: ${{ env.IMAGE_NAME }}
        IMAGE_TAG: ${{ github.event.inputs.image_tag }}
        HELM_AWS_ROLE_ARN: ${{ env.HELM_AWS_ROLE_ARN }}
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
      with:
        entrypoint: ./deploy/test-execute

Docker File

docker/Dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

# 安装 Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable

ARG ENVIRONMENT=docker

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./
RUN npm ci --no-optional

ENV ENVIRONMENT=
ENV DIRECTOR_ADDRESS=
ENV S3_REPORT_BUCKET_URL=
ENV S3_REPORT_BUCKET=
ENV AWS_REGION=
ENV RUN_ID=
ENV CI_BUILD_ID=
ENV TYPE=

CMD ENVIRONMENT=${ENVIRONMENT} AWS_REGION=${AWS_REGION} S3_REPORT_BUCKET_URL=${S3_REPORT_BUCKET_URL} S3_REPORT_BUCKET=${S3_REPORT_BUCKET} DIRECTOR_ADDRESS=${DIRECTOR_ADDRESS} RUN_ID=${RUN_ID} CI_BUILD_ID=${CI_BUILD_ID} TYPE=${TYPE} node ./cypress/scripts/cicd.js

docker/service-dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./

RUN npm ci --no-optional

CMD node ./cypress/service/service.js

Deploy

deploy/test-execute

bash 复制代码 
#!/bin/bash

# echo ${IDENTITY_TOKEN} > /tmp/token
set -e

unset AWS_PROFILE

export KUBECONFIG="/dev/shm/cicd"
aws --region ${AWS_REGION} eks update-kubeconfig --name=ndpg-${AWS_REGION_SHORT}-${STAGE}-eks-cluster

# kubectl complains if KUBECONFIG is world readable, so lock it down
chmod 600 "${KUBECONFIG}"

cd deploy

if [ "$RESEED_AT_DATA" = "only" ];then
  echo "find flag RESEED_AT_DATA is only, Skip run tests...."
  exit 0
fi

# Clean up the old secrets
kubectl get secrets --no-headers=true | awk '/cypress-automation-config/{print $1}'| xargs  kubectl delete secrets || true

# Deploy the director if none exists
if [ "$(kubectl get service -n ${KUBERNETES_NAMESPACE} ${RELEASE_NAME}-director -o jsonpath='{.metadata.name}')" != "${RELEASE_NAME}-director" ]; then
  echo -e "Starting director、dashboard、api and service...\n"
  helmfile -f helmfile-director.yaml apply --suppress-secrets
fi

# Run the initiator job
if [ ! ${RUN_ID} ]; then
  echo -e "Creating the TestRail run...\n"
else
  echo -e "Rerun tests that are not passed in the existing run, ${RUN_ID}...\n"
fi

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
INSTANCES=1 TYPE=initiator JOB_TIMEOUT=300 helmfile -f helmfile.yaml apply --suppress-secrets
#创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Complete --timeout=5m job/cypress-automation-${ENVIRONMENT}-initiator-${UNIQUE_ID}

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
echo -e "Starting workers...\n"
TYPE=worker JOB_TIMEOUT=9000 helmfile -f helmfile.yaml apply --suppress-secrets
echo -e "See the progress at https://pangaea.testrail.io/index.php?/runs/overview/5"
#创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Ready --timeout=10m -l job-name=cypress-automation-${ENVIRONMENT}-worker-${UNIQUE_ID} pods

echo -e "Done."

备注：

（1）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（2）创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}

（3）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（4）创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}

当Dockerfile.base生成的镜像在容器中运行时，会执行./cypress/scripts/cicd.js开始进入测试

当service-dockerfile.base生成的镜像在容器中运行时，会执行./cypress/service/service.js启动事件监听并在监听到执行完成事件时，向99u发送报告消息

其中，initiator和worker的区别在于，initiator会在testrail中创建新的run或者获取re-run的runId

相关推荐
陈希12330 分钟前
Linux 防火墙配置实战：firewalld、iptables 与云安全组
linux·运维·服务器
@兽血沸腾2 小时前
Windows系统安装Jenkins部署应用至远程Windows服务器操作指南
运维·服务器·jenkins
讯捷蓝达7 小时前
戴尔R720xd服务器卡死在 “Configuring Memory...Done“，换了内存还是卡？问题根本不在内存
运维·服务器
爱和冰阔落9 小时前
【Linux网络】TCP 服务器如何并发处理多个客户端？从 fork、僵尸进程到双重 fork
linux·运维·服务器·网络
lisw0510 小时前
基于神经网络的机器人导航系统的设计与执行！
神经网络·机器人·自动化
lisanmengmeng10 小时前
alertmanger 标签 rules 打上分组标签
linux·运维·服务器
严肃小白兔10 小时前
Cloudflare 更新 API Token 管理：从最小权限到自动化凭据安全
网络·安全·自动化
Android系统攻城狮11 小时前
Linux FFmpeg深度解析之av_write_frame调用流程与实战(三十三)
linux·运维·服务器·ffmpeg·音视频进阶·ffmpeg音视频
上火的金鱼妹11 小时前
k8s生产实际问题排查（网关，数据库读写等）
运维·服务器
孙克旭_11 小时前
k8s基础组件与创建资源
运维·docker·容器·kubernetes
热门推荐
01如何新建文件夹？ 电脑新建文件夹的4种方法02国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）03历年考研数学一、数学二、数学三真题试卷及答案PDF04GitHub 镜像站点05几个好用的ip纯净度检测网站06苹果iPA游戏软件资源下载网站07蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接08微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新09昇腾910B系列芯片完全指南：四款型号对比、服务器选型与大模型部署实战10Dify：一站式AI应用开发平台全解析与Windows部署指南