K8S Gateway AB测试、蓝绿发布、金丝雀(灰度)发布

假设有如下三个节点的 K8S 集群:

k8s31master 是控制节点

k8s31node1、k8s31node2 是工作节点

容器运行时是 containerd

一、场景分析

阅读本文,默认您已经安装了 K8S Gateway

关于 AB 测试、金丝雀发布,可以看这篇文章

二、实验准备

镜像下载

bash 复制代码
# 在各个工作节点下载
[root@k8s31node1 ~]# ctr -n=k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest
[root@k8s31node1 ~]# ctr -n=k8s.io images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest  docker.io/openresty/openresty:latest

[root@k8s31node2 ~]# ctr -n=k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest
[root@k8s31node2 ~]# ctr -n=k8s.io images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest  docker.io/openresty/openresty:latest

部署 v1

javascript 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:latest"
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v1
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v1
  name: nginx-v1
data:
  nginx.conf: |-
    worker_processes  1;
    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }
    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v1")
                ';
            }
        }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v1

该 yml 定义了三个资源 ConfigMap、Deployment、Service。

  • ConfigMap 定义了一个 nginx.conf 配置文件,使用 lua 脚本输出 nginx-v1。
  • Deployment 定义了一个 Pod,里面运行 openresty 它是一个封装了 nginx+lua 的 web 服务器。Pod 有两个标签 app: nginx、version: v1。
  • Service 代理了 Deployment 运行的 Pod。

部署 v2

javascript 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:latest"
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v2
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v2
  name: nginx-v2
data:
  nginx.conf: |-
    worker_processes  1;
    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }
    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v2")
                ';
            }
        }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v2

三、AB 测试

1)创建 gateway

javascript 复制代码
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: nginx-gateway
  namespace: default
spec:
  gatewayClassName: nginx
  listeners:
  - name: http
    protocol: HTTP
    port: 80

2)创建 httproute

javascript 复制代码
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: abtest-route
  namespace: default
spec:
  parentRefs:
    - name: nginx-gateway
  hostnames:
    - "abtest.example.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: / 
          headers:
            - name: "version"
              value: "v1"
      backendRefs:
        - name: nginx-v1
          kind: Service
          port: 80
    - matches:
        - path:
            type: PathPrefix
            value: /
          headers:
            - name: "version"
              value: "v2"  
      backendRefs:
        - name: nginx-v2
          kind: Service
          port: 80
  • parentRefs:绑定我们新建的 gateway。
  • hostnames:定义访问的主机名。
  • rules.matches:定义路由规则,PathPrefix 表示路径前缀匹配。headers 对请求头进行匹配。
  • backendRefs:定义后端服务以及服务端口。

3)测试

bash 复制代码
curl -H "Host: abtest.example.com" -H "version: v1" http://192.168.40.20:30185/
curl -H "Host: abtest.example.com" -H "version: v2" http://192.168.40.20:30185/

30185 为 nginx-gateway-controller 80 映射端口。

四、金丝雀发布

1)创建 httproute

javascript 复制代码
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-route
  namespace: default
spec:
  parentRefs:
    - name: nginx-gateway
  hostnames:
    - "canary.example.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: / 
      backendRefs:
        - name: nginx-v1
          kind: Service
          port: 80
          weight: 10
        - name: nginx-v2
          kind: Service
          port: 80
          weight: 90

backendRefs.weight 定义流量分发的权重。

2)测试

bash 复制代码
for i in {1..20}; do curl -H "Host: canary.example.com" http://192.168.40.20:30185/; done;
相关推荐
终端行者15 小时前
k8s之Ingress服务接入控制器
云原生·容器·kubernetes
学Linux的语莫20 小时前
k8s的nodeport和ingress
网络·rpc·kubernetes
云和数据.ChenGuang21 小时前
`neutron router-gateway-set` 操作失败的可能原因及解决方案
运维·gateway·运维技术总结·运维技术
aashuii1 天前
k8s通过NUMA亲和分配GPU和VF接口
云原生·容器·kubernetes
Most661 天前
kubesphere安装使用
kubernetes
Kentos(acoustic ver.)1 天前
云原生 —— K8s 容器编排系统
云原生·容器·kubernetes·云计算·k8s
哈里谢顿2 天前
Kubernetes 简介
kubernetes
__Smile°2 天前
k8s-MongoDB 副本集部署
云原生·容器·kubernetes
Jy_06222 天前
k8s 中的 deployment,statefulset,daemonset 控制器的区别
云原生·容器·kubernetes
果子⌂2 天前
Kubernetes 服务发布进阶
linux·运维·服务器·云原生·容器·kubernetes·云计算