Cypress自动化Github workflow

开水好喝2024-10-01 11:59

目录

Workflow

.github/workflows/build.yaml

jobs

build-deploy:

.github/wrokflows/execute-tests.yaml

输入参数

jobs

restart_failed_dp:

seed_data:

execute-tests:

[Docker File](#Docker File)

docker/Dockerfile.base

docker/service-dockerfile.base

Deploy

deploy/test-execute

Workflow

.github/workflows/build.yaml

创建docker image并推送到Amazon ECR

jobs

build-deploy:
bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Configure our environment based on the branch, or if this is a PR build
        if [[ '${{ github.ref }}' == refs/heads/${MAIN_BRANCH} ]]; then
            echo "::set-output name=tag::${{ github.sha }}"
            source .github/auto.env
            cat .github/auto.env >> ${GITHUB_ENV}
        else
          echo "This isn't either the main branch or a pull request. Exiting..."
          exit 1
        fi

        echo "::set-output name=kubernetes-namespace::${KUBERNETES_NAMESPACE}"
    - name: Build, tag, and push image to Amazon ECR
      env:
        IMAGE_TAG: ${{ steps.prep.outputs.tag }}
        GITHUB_TOKEN: ${{ secrets.ORG_TOKEN }}
      run: |
        # This is needed because the Github Action environment variables have issues if the value contains spaces (e.g. 'EXTRA_DOCKER_ARGS')
        source .github/global.env

        # Build and push the image using the SHA tag
        docker build -f ${{ env.DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${EXTRA_DOCKER_ARGS} .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG}

        docker build -f ${{ env.SERVICE_DOCKERFILE }} -t ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest .
        docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME_SERVICE }}:latest

        # Push this image with the 'latest' tag if this is a "main branch" build
        if [[ '${{ github.ref }}' == 'refs/heads/${{ env.MAIN_BRANCH }}' ]]; then
          docker tag ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:${IMAGE_TAG} ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
          docker push ${{ env.ECR_REGISTRY }}/${{ env.APP_NAME }}:latest
        fi

备注：

以上脚本执行了下列命令

复制代码 
docker build -f docker/Dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest ${EXTRA_DOCKER_ARGS} .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
复制代码 
docker build -f docker/service-dockerfile.base -t 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest .
docker push 590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation-service:latest

.github/wrokflows/execute-tests.yaml

输入参数

environment， 必填，选择一个测试环境，可选值为qa，staging等

platform，必填，选择哪一端的代码执行，可选值为web/android/ios等，主要是mobile的seed data也在该workflow中

instanceCount，必填，选择调用多少AWS集群机来执行本次任务

runId，选填，填入需要重跑的test run在testrails上面的id号

reseed_test_data，必填，选择是否执行seed data，销毁并重新生成站点测试数据

push_report，必填，选择是否发送测试报告到99u

image_tag，必填，选择是否获取代码库中的最新代码

jobs

restart_failed_dp:

执行./deploy/at_restart_failed_dp建立DP索引​​​

bash 复制代码 
runs-on: self-hosted
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        fi

        chmod 777 ./deploy/at_restart_failed_dp
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Restart Failed DP
      id: at_restart_failed_dp
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        AT_DP_PASSWORD: ${{ secrets.AT_DP_PASSWORD }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
      with:
        entrypoint: ./deploy/at_restart_failed_dp

备注：

  • global.env存储docker注册服务地址、AWS身份认证信息、docker名称、镜像名称、主干分支名、cypress服务端口等全局参数信息
  • auto.env存储AWS Region信息、K8S命名空间、AWS身份认证信息、IMAGE_TAG和自动化测试报告的S3存放url及目录信息
  • qa.env存储被测系统所在的AWS Region信息、AWS身份认证信息、API接口域名信息等环境参数
seed_data:

执行./deploy/at_data_setup调用one-eye的pod造AT所需数据

bash 复制代码 
runs-on: self-hosted
    needs: restart_failed_dp
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        if [[ "$ENVIRONMENT" == "dev" ]]; then
          source .github/dev.env
          cat .github/dev.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "nd" ]]; then
          source .github/nd.env
          cat .github/nd.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "qa" ]]; then
          source .github/qa.env
          cat .github/qa.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-staging" ]]; then
          source .github/sg-staging.env
          cat .github/sg-staging.env >> ${GITHUB_ENV}
        elif [[ "$ENVIRONMENT" == "sg-prod" ]]; then
          source .github/sg-prod.env
          cat .github/sg-prod.env >> ${GITHUB_ENV}
        else
          echo "Error env" $ENVIRONMENT
          exit 1
        fi
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume one-eye role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.ONE_EYE_HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AT Data Setup
      id: at_data_setup
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
        AT_DATA_POD_NAME: ${{ env.AT_DATA_POD_NAME }}
        STAGE: ${{ env.STAGE }}
        TEST_AWS_REGION: ${{ env.TEST_AWS_REGION }}
        TEST_AWS_REGION_SHORT: ${{ env.TEST_AWS_REGION_SHORT }}
        KUBERNETES_NAMESPACE: ${{ env.ONE_EYE_NS }}
        SEED_DATA_PASSWORD: ${{ env.SEED_DATA_PASSWORD }}
        PLATFORM: ${{ env.PLATFORM }}
      with:
        entrypoint: ./deploy/at_data_setup
execute-tests:

执行./deploy/test-execute启动K8S，分别启动initiator和worker进入测试执行

bash 复制代码 
runs-on: self-hosted
    timeout-minutes: 330
    needs: trigger_airflow
    steps:
    - name: "Checkout source code"
      uses: actions/checkout@v2
    - name: "Prepare environment"
      id: prep
      run: |
        # Source our global environment variables & add them to GITHUB_ENV for subsequent steps
        source .github/global.env
        cat .github/global.env >> ${GITHUB_ENV}

        # Source our environment variables for the "auto" account (where Automodo resides)
        source .github/auto.env
        cat .github/auto.env >> ${GITHUB_ENV}

        UNIQUE_ID=$(date +%s)
        echo UNIQUE_ID=$UNIQUE_ID >> $GITHUB_ENV

        IDENTITY_TOKEN=$(cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token)
        echo IDENTITY_TOKEN=${IDENTITY_TOKEN} >> $GITHUB_ENV
    - name: AWS assume cicd role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.GLOBAL_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: AWS assume helm role
      uses: aws-actions/configure-aws-credentials@v1
      env:
        AWS_SDK_LOAD_CONFIG: false
      with:
        aws-region: ${{ env.AWS_REGION }}
        role-to-assume: ${{ env.HELM_AWS_ROLE_ARN }}
        role-duration-seconds: 1200
        role-session-name: GitHubAction
    - name: Run Tests
      id: run-tests
      uses: docker://590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/infrastructure:latest
      env:
        STAGE: auto
        RELEASE_NAME: ${{ env.APP_NAME }}
        IMAGE_NAME: ${{ env.IMAGE_NAME }}
        IMAGE_TAG: ${{ github.event.inputs.image_tag }}
        HELM_AWS_ROLE_ARN: ${{ env.HELM_AWS_ROLE_ARN }}
        RESEED_AT_DATA: ${{ env.RESEED_AT_DATA }}
      with:
        entrypoint: ./deploy/test-execute

Docker File

docker/Dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

# 安装 Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable

ARG ENVIRONMENT=docker

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./
RUN npm ci --no-optional

ENV ENVIRONMENT=
ENV DIRECTOR_ADDRESS=
ENV S3_REPORT_BUCKET_URL=
ENV S3_REPORT_BUCKET=
ENV AWS_REGION=
ENV RUN_ID=
ENV CI_BUILD_ID=
ENV TYPE=

CMD ENVIRONMENT=${ENVIRONMENT} AWS_REGION=${AWS_REGION} S3_REPORT_BUCKET_URL=${S3_REPORT_BUCKET_URL} S3_REPORT_BUCKET=${S3_REPORT_BUCKET} DIRECTOR_ADDRESS=${DIRECTOR_ADDRESS} RUN_ID=${RUN_ID} CI_BUILD_ID=${CI_BUILD_ID} TYPE=${TYPE} node ./cypress/scripts/cicd.js

docker/service-dockerfile.base

bash 复制代码 
# --- Build/test ---
FROM cypress/browsers:node16.14.2-slim-chrome100-ff99-edge as base

WORKDIR /

# Use bash instead of sh for easier logic below
SHELL ["/bin/bash", "-c"]

ADD / ./

RUN npm ci --no-optional

CMD node ./cypress/service/service.js

Deploy

deploy/test-execute

bash 复制代码 
#!/bin/bash

# echo ${IDENTITY_TOKEN} > /tmp/token
set -e

unset AWS_PROFILE

export KUBECONFIG="/dev/shm/cicd"
aws --region ${AWS_REGION} eks update-kubeconfig --name=ndpg-${AWS_REGION_SHORT}-${STAGE}-eks-cluster

# kubectl complains if KUBECONFIG is world readable, so lock it down
chmod 600 "${KUBECONFIG}"

cd deploy

if [ "$RESEED_AT_DATA" = "only" ];then
  echo "find flag RESEED_AT_DATA is only, Skip run tests...."
  exit 0
fi

# Clean up the old secrets
kubectl get secrets --no-headers=true | awk '/cypress-automation-config/{print $1}'| xargs  kubectl delete secrets || true

# Deploy the director if none exists
if [ "$(kubectl get service -n ${KUBERNETES_NAMESPACE} ${RELEASE_NAME}-director -o jsonpath='{.metadata.name}')" != "${RELEASE_NAME}-director" ]; then
  echo -e "Starting director、dashboard、api and service...\n"
  helmfile -f helmfile-director.yaml apply --suppress-secrets
fi

# Run the initiator job
if [ ! ${RUN_ID} ]; then
  echo -e "Creating the TestRail run...\n"
else
  echo -e "Rerun tests that are not passed in the existing run, ${RUN_ID}...\n"
fi

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
INSTANCES=1 TYPE=initiator JOB_TIMEOUT=300 helmfile -f helmfile.yaml apply --suppress-secrets
#创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Complete --timeout=5m job/cypress-automation-${ENVIRONMENT}-initiator-${UNIQUE_ID}

#通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest
echo -e "Starting workers...\n"
TYPE=worker JOB_TIMEOUT=9000 helmfile -f helmfile.yaml apply --suppress-secrets
echo -e "See the progress at https://pangaea.testrail.io/index.php?/runs/overview/5"
#创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}
kubectl wait -n ${KUBERNETES_NAMESPACE} --for=condition=Ready --timeout=10m -l job-name=cypress-automation-${ENVIRONMENT}-worker-${UNIQUE_ID} pods

echo -e "Done."

备注：

（1）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（2）创建k8s容器如cypress-automation-qa-initiator-${UNIQUE_ID}

（3）通过执行helmfile.yaml拉取镜像文件590183924964.dkr.ecr.ap-southeast-1.amazonaws.com/cypress-automation:latest

（4）创建k8s容器如cypress-automation-qa-worker-${UNIQUE_ID}

当Dockerfile.base生成的镜像在容器中运行时，会执行./cypress/scripts/cicd.js开始进入测试

当service-dockerfile.base生成的镜像在容器中运行时，会执行./cypress/service/service.js启动事件监听并在监听到执行完成事件时，向99u发送报告消息

其中，initiator和worker的区别在于，initiator会在testrail中创建新的run或者获取re-run的runId

相关推荐
VinciYan32 分钟前
基于Jenkins+Docker的自动化部署实践——整合Git与Python脚本实现远程部署
python·ubuntu·docker·自动化·jenkins·.net·运维开发
冷曦_sole1 小时前
linux-19 根文件系统(一）
linux·运维·服务器
AI大模型学徒1 小时前
Linux(二)_清理空间
linux·运维·服务器
tntlbb1 小时前
Ubuntu20.4 VPN+Docker代理配置
运维·ubuntu·docker·容器
一条测试老狗2 小时前
【UI自动化】从WebDriver看Selenium与Appium的底层关联
selenium·appium·自动化
Linux运维技术栈3 小时前
Ansible(自动化运维)环境搭建及ansible-vault加密配置
运维·自动化·ansible
Bessssss4 小时前
centos权限大集合，覆盖多种权限类型，解惑权限后有“. + t s”问题！
linux·运维·centos
苹果醋34 小时前
Golang的文件加密工具
运维·vue.js·spring boot·nginx·课程设计
jwensh4 小时前
【Jenkins】Declarative和Scripted两种脚本模式有什么具体的区别
运维·前端·jenkins
大熊程序猿5 小时前
xxl-job docker 安装
运维·docker·容器
热门推荐
01【一文读懂】NTN（非地面网络）技术介绍02优化手机性能，解决卡顿问题：关闭这3个微信开关，释放内存空间03玄机平台应急响应—webshell查杀04Cube-Studio：开源大模型全链路一站式中台05基于大语言模型（LLM）的合成数据生成、策展和评估的综述06(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明0719个Web前端交互式3D JavaScript框架和库08HCIA-datacom数通题库和录播视频资料09Docker 夺命连环 15 问10Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO