K8S Ingress-Nginx导出TCP端口

ingress-nginx导出TCP端口

• Exposing TCP and UDP services - Ingress-Nginx Controller
• https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md

helm upgrade ingress-nginx导出redis 6379端口（这种方式最简单，接下为我们研究一下主要的流程原理）

• • K8s and Redis; a tale of Layer 4 Ingress

    helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx
    --values values.yaml
    --set tcp.26379="develop/redis-cluster:6379" --debug

• nginx-ingress 已经能够完成 7/4 层的代理功能（4 层代理基于 ConfigMap）

• 流程图，请参考：Kubernetes K8S 1.20部署Ingress nginx 0.30_k8s1.20.9部署ingress-nginx-CSDN博客

整体流程大概是这样子的

• SVC(NodePort:30009映射到后台26379端口，26379端口转发到26379-tcp)
  • DaemonSet/ingress-nginx-controller(26379-tcp转到controller容器的端口containerPort: 26379)
    • ingress-nginx-controller容器中有nginx，nginx监听26379端口
      • 从configMap中取的26379端口要转发的地址
        • develop/redis-cluster:6379
• 用户请求流程
  • 访问30009NodePort端口
    • 映射到26379端口
      • ingress-nginx-controller中的nginx监听26379端口
        • 将26379端口转发至develop/redis-cluster:6379

添加configMap

vi configMap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  26379: "develop/redis-cluster:6379" #从26379的TCP端口转发到Redis集群的6379端口，至于nodePort是哪个还未定

kubectl apply -f configMap.yaml -n ingress-nginx

也可以自己配置SVC

编辑service, 即ClusterIP
  kubectl edit svc -n ingress-nginx ingress-nginx-controller
  - name: 26379-tcp #配置一个新的TCP端口，取名为26379-tcp
    nodePort: 30009 #导出对外的端口
    port: 26379 #监听26379端口
    protocol: TCP
    targetPort: 26379-tcp #转发到ds上的26379-tcp端口

配置DaemonSet

• ingress-nginx-controller pod实际是Controlled By: DaemonSet/ingress-nginx-controller

• 修改DaemonSet/ingress-nginx-controller

  • 将svc上监听到的26379端口转发到DaemonSet/ingress-nginx-controller上

• DaemonSet/ingress-nginx-controller连接到ingress-nginx-controller对应的pod容器的26379端口上

  kubectl edit ds -n ingress-nginx ingress-nginx-controller

  • --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services

     ports:
     - containerPort: 80
       name: http
       protocol: TCP
     - containerPort: 443
       name: https
       protocol: TCP
     - containerPort: 8443
       name: webhook
       protocol: TCP
     - containerPort: 26379
       name: 26379-tcp
       protocol: TCP

  #查看SVC，已经有了26379端口，并绑定了nodePort端口30009
  [root@k8s-master01 ingress-nginx]# kubectl get svc -n ingress-nginx
  NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  ingress-nginx-controller LoadBalancer 10.99.163.44 <pending> 80:31268/TCP,443:31052/TCP,26379:30009/TCP 3d11h
  ingress-nginx-controller-admission ClusterIP 10.100.131.12 <none> 443/TCP

查看nginx.conf代理配置

• kubectl exec -it ingress-nginx-controller-lkzzh /bin/sh -n ingress-nginx

  • /etc/nginx
  • $ ls -l
  • cat nginx.conf

• 监听26379端口，应该是configMap中的data的26379

  • 从configMap中取出26379端口要转发的地址

• 转发到tcp-develop-redis-cluster-6379

  TCP services

        server {
                preread_by_lua_block {
                        ngx.var.proxy_upstream_name="tcp-develop-redis-cluster-6379";
                }
  
                listen                  26379;
  
                listen                  [::]:26379;
  
                proxy_timeout           600s;
                proxy_next_upstream     on;
                proxy_next_upstream_timeout 600s;
                proxy_next_upstream_tries   3;
  
                proxy_pass              upstream_balancer;
  
        }

测试

• telnet可以通过
• telnet 192.168.221.131 30009