Cypress自动化Github workflow

开水好喝2024-10-01 11:59

目录

Workflow

.github/workflows/build.yaml

jobs

build-deploy:

.github/wrokflows/execute-tests.yaml

输入参数

jobs

restart_failed_dp:

seed_data:

execute-tests:

[Docker File](#Docker File)

docker/Dockerfile.base

docker/service-dockerfile.base

Deploy

deploy/test-execute

Workflow

.github/workflows/build.yaml

创建docker image并推送到Amazon ECR

jobs

build-deploy:
bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Configure our environment based on the branch, or if this is a PR build
        if [[ '${{ github.ref }}' == refs/heads/${MAIN_BRANCH} ]]; then
            echo "::set-output name=tag::${{ github.sha }}"
            source .github/auto.env
            cat .github/auto.env >> ${GITHUB_ENV}
        else
          echo "This isn't either the main branch or a pull request. Exiting..."
          exit 1
        fi

        echo "::set-output name=kubernetes-namespace::${KUBERNETES_NAMESPACE}"
    - name: Build, tag, and push image to Amazon ECR
      env:
        IMAGE_TAG: ${{ steps.prep.outputs.tag }}
        GITHUB_TOKEN: ${{ secrets.ORG_TOKEN }}
      run: |
        # This is needed because the Github Action environment variables have issues if the value contains spaces (e.g. 'EXTRA_DOCKER_ARGS')
        source .github/global.env

        # Build and push the image using the SHA tag
        docker build -f ${{ env.DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${EXTRA_DOCKER_ARGS} .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG}

        docker build -f ${{ env.SERVICE_DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest

        # Push this image with the 'latest' tag if this is a "main branch" build
        if [[ '${{ github.ref }}' == 'refs/heads/${{ env.MAIN_BRANCH }}' ]]; then
          docker tag ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
          docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
        fi

备注：

以上脚本执行了下列命令

复制代码 
docker build -f docker/Dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest ${EXTRA_DOCKER_ARGS} .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
复制代码 
docker build -f docker/service-dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest

.github/wrokflows/execute-tests.yaml

输入参数

environment， 必填，选择一个测试环境，可选值为qa，staging等

platform，必填，选择哪一端的代码执行，可选值为web/android/ios等，主要是mobile的seed data也在该workflow中

instanceCount，必填，选择调用多少AWS集群机来执行本次任务

runId，选填，填入需要重跑的test run在testrails上面的id号

reseed_test_data，必填，选择是否执行seed data，销毁并重新生成站点测试数据

push_report，必填，选择是否发送测试报告到99u

image_tag，必填，选择是否获取代码库中的最新代码

jobs

restart_failed_dp:

执行./deploy/at_restart_failed_dp建立DP索引​​​

bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        fi

        chmod 777 ./deploy/at_restart_failed_dp
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Restart Failed DP
      id: at_restart_failed_dp
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        AT_DP_PASSWORD: ${{ secrets.AT_DP_PASSWORD }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
      with:
        entrypoint: ./deploy/at_restart_failed_dp

备注：

  • global.env存储docker注册服务地址、AWS身份认证信息、docker名称、镜像名称、主干分支名、cypress服务端口等全局参数信息
  • auto.env存储AWS Region信息、K8S命名空间、AWS身份认证信息、IMAGE_TAG和自动化测试报告的S3存放url及目录信息
  • qa.env存储被测系统所在的AWS Region信息、AWS身份认证信息、API接口域名信息等环境参数
seed_data:

执行./deploy/at_data_setup调用one-eye的pod造AT所需数据

bash 复制代码 
runs-on: self-hosted
    needs: restart_failed_dp
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "dev" ]]; then
          source .github/dev.env
          cat .github/dev.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "nd" ]]; then
          source .github/nd.env
          cat .github/nd.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        else
          echo "Error env" $ENVIRONMENT
          exit 1
        fi
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Data Setup
      id: at_data_setup
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
        AT_DATA_POD_NAME: ${{ env.AT_DATA_POD_NAME }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
        SEED_DATA_PASSWORD: ${{ env.SEED_DATA_PASSWORD }}
        PLATFORM: ${{ env.PLATFORM }}
      with:
        entrypoint: ./deploy/at_data_setup
execute-tests:

执行./deploy/test-execute启动K8S，分别启动initiator和worker进入测试执行

bash 复制代码 
runs-on: self-hosted
    timeout-minutes: 330
    needs: trigger_airflow
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Source our environment variables for the "auto" account (where Automodo resides)
        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        UNIQUE_ID=$(date +%s)
        echo UNIQUE_ID=$UNIQUE_ID >> $GITHUB_ENV

        IDENTITY_TOKEN=$(cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token)
        echo IDENTITY_TOKEN=${IDENTITY_TOKEN} >> $GITHUB_ENV
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume helm role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: Run Tests
      id: run-tests
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        STAGE: auto
        RELEASE_NAME: ${{ env.APP_NAME }}
        IMAGE_NAME: ${{ env.IMAGE_NAME }}
        IMAGE_TAG: ${{ github.event.inputs.image_tag }}
        HELM_AWS_ROLE_ARN: ${{ env.HELM_AWS_ROLE_ARN }}
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
      with:
        entrypoint: ./deploy/test-execute

Docker File

docker/Dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

# 安装 Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable

ARG ENVIRONMENT=docker

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./
RUN npm ci --no-optional

ENV ENVIRONMENT=
ENV DIRECTOR_ADDRESS=
ENV S3_REPORT_BUCKET_URL=
ENV S3_REPORT_BUCKET=
ENV AWS_REGION=
ENV RUN_ID=
ENV CI_BUILD_ID=
ENV TYPE=

CMD ENVIRONMENT=${ENVIRONMENT} AWS_REGION=${AWS_REGION} S3_REPORT_BUCKET_URL=${S3_REPORT_BUCKET_URL} S3_REPORT_BUCKET=${S3_REPORT_BUCKET} DIRECTOR_ADDRESS=${DIRECTOR_ADDRESS} RUN_ID=${RUN_ID} CI_BUILD_ID=${CI_BUILD_ID} TYPE=${TYPE} node ./cypress/scripts/cicd.js

docker/service-dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./

RUN npm ci --no-optional

CMD node ./cypress/service/service.js

Deploy

deploy/test-execute

bash 复制代码 
#!/bin/bash

# echo ${IDENTITY_TOKEN} > /tmp/token
set -e

unset AWS_PROFILE

export KUBECONFIG="/dev/shm/cicd"
aws --region ${AWS_REGION} eks update-kubeconfig --name=ndpg-${AWS_REGION_SHORT}-${STAGE}-eks-cluster

# kubectl complains if KUBECONFIG is world readable, so lock it down
chmod 600 "${KUBECONFIG}"

cd deploy

if [ "$RESEED_AT_DATA" = "only" ];then
  echo "find flag RESEED_AT_DATA is only, Skip run tests...."
  exit 0
fi

# Clean up the old secrets
kubectl get secrets --no-headers=true | awk '/cypress-automation-config/{print $1}'| xargs  kubectl delete secrets || true

# Deploy the director if none exists
if [ "$(kubectl get service -n ${KUBERNETES_NAMESPACE} ${RELEASE_NAME}-director -o jsonpath='{.metadata.name}')" != "${RELEASE_NAME}-director" ]; then
  echo -e "Starting director、dashboard、api and service...\n"
  helmfile -f helmfile-director.yaml apply --suppress-secrets
fi

# Run the initiator job
if [ ! ${RUN_ID} ]; then
  echo -e "Creating the TestRail run...\n"
else
  echo -e "Rerun tests that are not passed in the existing run, ${RUN_ID}...\n"
fi

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
INSTANCES=1 TYPE=initiator JOB_TIMEOUT=300 helmfile -f helmfile.yaml apply --suppress-secrets
#创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Complete --timeout=5m job/cypress-automation-${ENVIRONMENT}-initiator-${UNIQUE_ID}

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
echo -e "Starting workers...\n"
TYPE=worker JOB_TIMEOUT=9000 helmfile -f helmfile.yaml apply --suppress-secrets
echo -e "See the progress at https://pangaea.testrail.io/index.php?/runs/overview/5"
#创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Ready --timeout=10m -l job-name=cypress-automation-${ENVIRONMENT}-worker-${UNIQUE_ID} pods

echo -e "Done."

备注：

（1）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（2）创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}

（3）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（4）创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}

当Dockerfile.base生成的镜像在容器中运行时，会执行./cypress/scripts/cicd.js开始进入测试

当service-dockerfile.base生成的镜像在容器中运行时，会执行./cypress/service/service.js启动事件监听并在监听到执行完成事件时，向99u发送报告消息

其中，initiator和worker的区别在于，initiator会在testrail中创建新的run或者获取re-run的runId

相关推荐
打码人的日常分享6 分钟前
商用密码应用安全性评估，密评整体方案，密评管理测评要求和指南，运维文档，软件项目安全设计相关文档合集（Word原件）
运维·安全·web安全·系统安全·规格说明书
追风赶月、1 小时前
【Linux】线程概念与线程控制
linux·运维·服务器
CP-DD1 小时前
Docker 容器化开发 应用
运维·docker·容器
努力的悟空3 小时前
国土变更调查拓扑错误自动化修复工具的研究
运维·自动化
周末不下雨4 小时前
win11+ubuntu22.04双系统 | 联想 24 y7000p | ubuntu 22.04 | 把ubuntu系统装到1T的移动固态硬盘上！！！
linux·运维·ubuntu
耗同学一米八5 小时前
2024 年河北省职业院校技能大赛网络建设与运维赛项样题四
运维·网络
东华果汁哥5 小时前
【linux 免密登录】快速设置kafka01、kafka02、kafka03 三台机器免密登录
linux·运维·服务器
肖永威6 小时前
CentOS环境上离线安装python3及相关包
linux·运维·机器学习·centos
zhd15306915625ff6 小时前
库卡机器人维护需要注意哪些事项
安全·机器人·自动化
布鲁格若门6 小时前
CentOS 7 桌面版安装 cuda 12.4
linux·运维·centos·cuda
热门推荐
01(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明02PyTorch机器学习实现液态神经网络03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04玄机平台应急响应—webshell查杀05Coze扣子平台完整体验和实践（附国内和国际版对比）06Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO07怎样让音频速度变慢？请跟随以下方法进行操作08Unity中PICO实现 隔空取物 和 接触抓取物体09RAG 实践- Ollama+RagFlow 部署本地知识库10【目标跟踪】相机运动补偿