【k8s】ingress-nginx通过header路由到不同后端

背景

公司使用ingress-nginx作为网关的项目，需要在相同域名、uri，根据header将请求转发到不同的后端中
在稳定发布的情况下，ingress-nginx是没有语法直接支持根据header做转发的。但是这个可以利用灰度发布的特性实现header路由功能

准备

准备两个后端，后端代码如下，路由均为 /app
- main.go

go 复制代码

package main

import "github.com/gin-gonic/gin"

func main() {
	r := gin.Default()
	r.GET("/app", func(context *gin.Context) {
		context.JSON(200, gin.H{"message": "app1"})
	})
	r.Run(":8080")
}

使用Dockerfile构建镜像
- 这里构建 goapp1:v1，goapp2:v1两个镜像（goapp2请将main.go修改 "message": "app2"）
  
  FROM golang:1.17.13
  RUN mkdir -p /go/app/;
  cd /go/app/;
  go mod init app1;
  GOPROXY="https://goproxy.cn,direct" go get github.com/gin-gonic/gin@v1.6.3
  WORKDIR /go/app/
  COPY main.go /go/app
  EXPOSE 8080
  CMD go run main.go

此解决方案参考：https://v2-1.docs.kubesphere.io/docs/zh-CN/quick-start/ingress-canary/
注：本人使用低版本ingress-nginx，高版本的请大家自行修改不同之处
首先部署goapp1:v1 和 goapp2:v1 的deployment和service
- 此为goapp1。goapp2请自行修改

yaml 复制代码

apiVersion: apps/v1
kind: Deployment
metadata:
  name: goapp1
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: goapp1
  template:
    metadata:
      labels:
        app: goapp1
    spec:
      containers:
      - image: goapp1:v1
        imagePullPolicy: IfNotPresent
        name: goapp1
        ports:
        - containerPort: 80
          protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: goapp1
  namespace: default
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: goapp1

部署稳定发布版本的ingress，路由至goapp1

yaml 复制代码

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: goapp1 
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /app 
        pathType: Prefix
        backend:
          service:
            name: goapp1
            port: 
              number: 8080

部署canary版本的ingress，路由至goapp2
- 这里可见域名都是 test.com，uri都是 /app
- 注解：
  - nginx.ingress.kubernetes.io/canary: "true" # 启用canary灰度发布特性
  - nginx.ingress.kubernetes.io/canary-by-header: canary # 通过header可选择是否转发至canary版本的后端

yaml 复制代码

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: goapp2
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: canary
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /app 
        pathType: Prefix
        backend:
          service:
            name: goapp2
            port: 
              number: 8080

进行测试

bash 复制代码

for i in {1..20};
  # ingress-nginx的NodePort请自行查看，替换下面的端口
  do curl test.com:31132/app -H "canary: never"; # 路由至稳定版本的goapp1
  echo -e "";
done

bash 复制代码

for i in {1..20};
  do curl test.com:31132/app -H "canary: always"; # 路由至canary版本的goapp2
  echo -e "";
done

效果如下，可以看到可以通过header控制发送请求到不同后端，能够满足需求

通过nginx进行转发

第二种方法可通过在k8s集群部署一个nginx，通过nginx进行分流
- 流量路径如下： ingress-nginx --> nginx --> goapp1或goapp2
这里nginx写法有比较多，我选择最简单的通过if判断$http_my_header
在使用$http_my_header之前，需要对ingress-nginx和nginx添加参数，允许header中存在下划线
- ingress-nginx

bash 复制代码

kubectl edit cm ingress-nginx-controller
------------------
apiVersion: v1
data:
  allow-snippet-annotations: "true"
  # 添加下面这两个参数
  enable-underscores-in-headers: "true"
  ignore-invalid-headers: "false"
kind: ConfigMap

部署nginx，nginx中开启允许header下划线的参数：underscores_in_headers on;

yaml 复制代码

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.24.0
        ports:
        - containerPort: 80
        volumeMounts:
          - name: nginx
            mountPath: /etc/nginx/nginx.conf
            subPath: nginx.conf
      volumes:
        - name: nginx
          configMap:
            name: nginx
            items:
            - key: nginx.conf
              path: nginx.conf
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: v1
data:
  nginx.conf: |
    user  nginx;
    worker_processes  auto;
    
    error_log  /var/log/nginx/error.log notice;
    pid        /var/run/nginx.pid;
    
    
    events {
        worker_connections  1024;
    }
    
    
    http {
    
        upstream upstream_server1 {
            server goapp1:8080;
        }
    
        upstream upstream_server2 {
            server goapp2:8080;
        }
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;
    
        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for" "$http_my_header"';
    
        access_log  /var/log/nginx/access.log  main;
    
        sendfile        on;
    
        keepalive_timeout  65;
    
        server {
            underscores_in_headers on; 
            listen 80;
            server_name test.com;
    
            location /app {
              if ($http_my_header = "value1") {
                  proxy_pass http://upstream_server1;
              }
              if ($http_my_header = "value2") {
                  proxy_pass http://upstream_server2;
              }
            }
        }
    
    }

kind: ConfigMap
metadata:
  name: nginx
  namespace: default

上面的配置判断 $http_my_header是 value1 还是 value2，再转发到不同的upstream
测试

bash 复制代码

curl test.com/app -H "my_header:value1"
curl test.com/app -H "my_header:value2"

【k8s】ingress-nginx通过header路由到不同后端

K8S中ingress-nginx通过header路由到不同后端

背景

准备

使用灰度发布的特性进行header的路由

通过nginx进行转发