K8S Nginx Ingress实现金丝雀发布

summer_west_fish2023-08-29 10:07

通过给 Ingress 资源指定 Nginx Ingress 所支持的 annotation 可实现金丝雀发布。

需给服务创建2个 Ingress，其中1个常规 Ingress，另1个为带 nginx.ingress.kubernetes.io/canary: "true" 固定的 annotation 的 Ingress，称为 Canary Ingress。

Canary Ingress 一般代表新版本的服务，结合另外针对流量切分策略的 annotation 一起配置即可实现多种场景的金丝雀发布。

以下为相关 annotation 的详细介绍：

  • nginx.ingress.kubernetes.io/canary-by-header
    表示如果请求头中包含指定的 header 名称，并且值为 always，就将该请求转发给该 Ingress 定义的对应后端服务。如果值为 never 则不转发，可以用于回滚到旧版。如果为其他值则忽略该 annotation。

  • nginx.ingress.kubernetes.io/canary-by-header-value

    该 annotation 可以作为 canary-by-header 的补充，可指定请求头为自定义值，包含但不限于 always 或 never。当请求头的值命中指定的自定义值时，请求将会转发给该 Ingress 定义的对应后端服务，如果是其它值则忽略该 annotation。

  • nginx.ingress.kubernetes.io/canary-by-header-pattern

    与 canary-by-header-value 类似，区别为该 annotation 用正则表达式匹配请求头的值，而不是只固定某一个值。如果该 annotation 与 canary-by-header-value 同时存在，该 annotation 将被忽略。

  • nginx.ingress.kubernetes.io/canary-by-cookie

    与 canary-by-header 类似，该 annotation 用于 cookie，仅支持 always 和 never。

  • nginx.ingress.kubernetes.io/canary-weight

    表示 Canary Ingress 所分配流量的比例的百分比，取值范围 [0-100]。例如，设置为10，则表示分配10%的流量给 Canary Ingress 对应的后端服务。

一、部署蓝环境版本服务

1、ConfigMap

bash 复制代码 
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-blue-config
data:
  nginx.conf: |-
    worker_processes  1;

    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }

    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("blue")
                ';
            }
        }
    }

2、Deployment

bash 复制代码 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-blue
  labels:
    dce.daocloud.io/app: nginx-blue
  annotations:
    dce.daocloud.io/last-replicas: '1'
    deployment.kubernetes.io/revision: '3'
    kubernetes.io/change-cause: update YAML
spec:
  replicas: 1
  selector:
    matchLabels:
      dce.daocloud.io/component: nginx-blue
  template:
    metadata:
      name: nginx-blue
      labels:
        dce.daocloud.io/app: nginx-blue
        dce.daocloud.io/component: nginx-blue
      annotations:
        dce.daocloud.io/parcel.egress.burst: '0'
        dce.daocloud.io/parcel.egress.rate: '0'
        dce.daocloud.io/parcel.ingress.burst: '0'
        dce.daocloud.io/parcel.ingress.rate: '0'
        dce.daocloud.io/parcel.net.type: calico
    spec:
      volumes:
        - name: nginx-blue-config
          configMap:
            name: nginx-blue-config
            defaultMode: 420
      containers:
        - name: nginx-blue
          image: 'x.x.x.x/library/openresty:1.19.9.1-sw-r4'
          resources:
            limits:
              cpu: 500m
              memory: '314572800'
            requests:
              cpu: 200m
              memory: '314572800'
          volumeMounts:
            - name: nginx-blue-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf

3、Service

bash 复制代码 
kind: Service
apiVersion: v1
metadata:
  name: nginx-blue-default
  labels:
    dce.daocloud.io/app: nginx-blue
  annotations:
    io.daocloud.dce.serviceSelectorType: service
spec:
  ports:
    - name: nginx-nginx-default-80680-80
      protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 31046
  selector:
    dce.daocloud.io/component: nginx-blue
  clusterIP: 172.31.69.137
  type: NodePort
  sessionAffinity: None
  externalTrafficPolicy: Cluster

4、修改pod内容

bash 复制代码 
cd /usr/local/openresty/nginx/html/

ls
50x.html  index.html

echo "Hello Blue" > index.html

cat index.html 
Hello Blue

二、部署绿环境版本服务

1、ConfigMap

bash 复制代码 
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-green-config
data:
  nginx.conf: |-
    worker_processes  1;

    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }

    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("green")
                ';
            }
        }
    }

2、Deployment

bash 复制代码 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-green
  labels:
    dce.daocloud.io/app: nginx-green
  annotations:
    deployment.kubernetes.io/revision: '5'
    kubernetes.io/change-cause: update YAML
spec:
  replicas: 1
  selector:
    matchLabels:
      dce.daocloud.io/component: nginx-green
  template:
    metadata:
      name: nginx-green
      labels:
        dce.daocloud.io/app: nginx-green
        dce.daocloud.io/component: nginx-green
        env: green
      annotations:
        dce.daocloud.io/parcel.egress.burst: '0'
        dce.daocloud.io/parcel.egress.rate: '0'
        dce.daocloud.io/parcel.ingress.burst: '0'
        dce.daocloud.io/parcel.ingress.rate: '0'
        dce.daocloud.io/parcel.net.type: calico
        dce.daocloud.io/parcel.net.value: default-ipv4-ippool
    spec:
      volumes:
        - name: nginx-green-config
          configMap:
            name: nginx-green-config
            defaultMode: 420
      containers:
        - name: nginx-green
          image: 'x.x.x.x/library/openresty:1.19.9.1-sw-r4'
          resources:
            limits:
              cpu: 500m
              memory: '314572800'
            requests:
              cpu: 200m
              memory: '314572800'
          volumeMounts:
            - name: nginx-green-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf

3、Service

bash 复制代码 
kind: Service
apiVersion: v1
metadata:
  name: nginx-green-default
  labels:
    dce.daocloud.io/app: nginx-green
  annotations:
    io.daocloud.dce.serviceSelectorType: service
spec:
  ports:
    - name: nginx-nginx-default-15833-80
      protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 35218
  selector:
    dce.daocloud.io/component: nginx-green
  clusterIP: 172.31.207.22
  type: NodePort
  sessionAffinity: None
  externalTrafficPolicy: Cluster

4、修改pod内容

bash 复制代码 
cd /usr/local/openresty/nginx/html/

ls
50x.html  index.html

echo "Hello Green" > index.html

cat index.html 
Hello Green

三、设置Ingress

1、blue环境Ingress

bash 复制代码 
kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: nginx-blue-ingress
  labels:
    dce.daocloud.io/app: nginx-blue
  annotations:
    nginx.ingress.kubernetes.io/use-port-in-redirects: 'true'
spec:
  rules:
    - host: nginx.ms-sit.xxxxxx.net
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              serviceName: nginx-blue-default
              servicePort: 80

2、green环境Ingress

bash 复制代码 
kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: nginx-green-ingress
  labels:
    dce.daocloud.io/app: nginx-green
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: 'true'
    nginx.ingress.kubernetes.io/canary-by-header: env
    nginx.ingress.kubernetes.io/canary-by-header-pattern: green
spec:
  rules:
    - host: nginx.ms-sit.xxxxxx.net
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              serviceName: nginx-green-default
              servicePort: 80

四、测试

相关推荐
Gold Steps.2 小时前
OpenEBS — 云原生 CNS 高性能存储
云原生·kubernetes·存储
larance6 小时前
Gunicorn + Nginx+systemd 配置flask
nginx·flask·gunicorn
文艺理科生Owen7 小时前
Nginx 路径映射深度解析：从本地开发到生产交付的底层哲学
运维·nginx
广州中轴线9 小时前
OpenStack on Kubernetes 生产部署实战（十三）
容器·kubernetes·openstack
切糕师学AI10 小时前
Helm Chart 是什么？
云原生·kubernetes·helm chart
广州中轴线12 小时前
OpenStack on Kubernetes 生产部署实战（十七）
容器·kubernetes·openstack
kong790692813 小时前
Nginx性能优化
java·nginx·性能优化
Harvey9031 天前
通过 Helm 部署 Nginx 应用的完整标准化步骤
linux·运维·nginx·k8s
陈桴浮海1 天前
Kustomize实战：从0到1实现K8s多环境配置管理与资源部署
云原生·容器·kubernetes
张小凡vip1 天前
Kubernetes--k8s中部署redis数据库服务
redis·kubernetes
热门推荐
01GitHub 镜像站点02Claude Code + GLM4.7 避坑指南：解决 Unable to connect to Anthropic services03openclaw配置教程（linux+局域网ollama）04UV安装并设置国内源05Linux下V2Ray安装配置指南06AI 规范驱动开发“三剑客”深度对比：Spec-Kit、Kiro 与 OpenSpec 实战指南07openclaw使用nginx反代部署过程 与disconnected (1008): pairing required解决08Claude Code Skills 实用使用手册09在Trae中使用Pencil MCP10Vue-skills的中文文档