k8s-ingress控制器

一、概念
[二、安装ingress controller](#二、安装ingress controller)
- [2.1 将镜像scp到所有的node节点服务器上](#2.1 将镜像scp到所有的node节点服务器上)
- [2.2 导入镜像，在所有的节点服务器上进行](#2.2 导入镜像，在所有的节点服务器上进行)
- [2.3 使用ingress-controller-deploy.yaml 文件去启动ingress controller](#2.3 使用ingress-controller-deploy.yaml 文件去启动ingress controller)
- [2.4 查看ingress controller的相关信息](#2.4 查看ingress controller的相关信息)
- [2.5 创建pod和暴露pod的服务](#2.5 创建pod和暴露pod的服务)
- [2.6 启用ingress 关联ingress controller 和service](#2.6 启用ingress 关联ingress controller 和service)
- [2.7 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则](#2.7 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则)
- [2.8 进入ingress controller对应的pod里查看nginx.conf的配置](#2.8 进入ingress controller对应的pod里查看nginx.conf的配置)
二、基于url的负载均衡的实现

官方文档：https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/

一、概念

Ingress 控制器是实现外部流量（如来自互联网的 HTTP/HTTPS 请求）路由到集群内部服务的核心组件。它解决了传统 Service（如 NodePort、LoadBalancer）在多服务、多域名场景下的灵活性不足问题，提供了统一的入口管理、域名路由、SSL 终止等高级功能

Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求

实际上，Ingress相当于一个7层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解成在Ingress里建立诸多映射规则，Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务

ingress：kubernetes中的一个对象，作用是定义请求如何转发到service的规则
ingress controller：具体实现反向代理及负载均衡的程序，对ingress定义的规则进行解析，根据配置的规则来实现请求转发

二、安装ingress controller

2.1 将镜像scp到所有的node节点服务器上

$root@k8s-1 ingress\]# scp ingress-nginx-controllerv1.1.0.tar.gz k8s-2:/root \[root@k8s-1 ingress\]# scp ingress-nginx-controllerv1.1.0.tar.gz k8s-3:/root \[root@k8s-1 ingress\]# scp kube-webhook-certgen-v1.1.0.tar.gz k8s-2:/root \[root@k8s-1 ingress\]# scp kube-webhook-certgen-v1.1.0.tar.gz k8s-3:/root$

2.2 导入镜像，在所有的节点服务器上进行

bash 复制代码

[root@k8s-2 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz 
[root@k8s-2 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz 
[root@k8s-2 ~]# docker images
REPOSITORY                                                                     TAG        IMAGE ID       CREATED         SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller   v1.1.0     ae1a7201ec95   3 years ago     285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen       v1.1.1     c41e9fcadf5a   3 years ago     47.7MB

2.3 使用ingress-controller-deploy.yaml 文件去启动ingress controller

bash 复制代码

[root@k8s-1 ingress]# kubectl apply -f ingress-controller-deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

2.4 查看ingress controller的相关信息

bash 复制代码

[root@k8s-1 ingress]# kubectl get ns|grep ingress
NAME                   STATUS   AGE
ingress-nginx          Active   32s
[root@k8s-1 ingress]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.105.212.102   <none>        80:31407/TCP,443:32222/TCP   46s
ingress-nginx-controller-admission   ClusterIP   10.106.179.205   <none>        443/TCP                      46s
[root@k8s-1 ingress]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-rq5cg        0/1     Completed   0          58s
ingress-nginx-admission-patch-hchjm         0/1     Completed   0          58s
ingress-nginx-controller-7cd558c647-hmzmx   1/1     Running     0          58s
ingress-nginx-controller-7cd558c647-w867n   1/1     Running     0          58s

2.5 创建pod和暴露pod的服务

bash 复制代码

[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-1.yaml 
deployment.apps/sc-nginx-deploy created
service/sc-nginx-svc created
[root@k8s-1 ingress]# kubectl get deploy
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
sc-nginx-deploy        3/3     3            3           13s
[root@k8s-1 ingress]# kubectl get pod
NAME                                    READY   STATUS      RESTARTS       AGE
sc-nginx-deploy-55cc5ffddf-2tl7b        1/1     Running     0              16s
sc-nginx-deploy-55cc5ffddf-crdhm        1/1     Running     0              16s
sc-nginx-deploy-55cc5ffddf-xxkwt        1/1     Running     0              16s

[root@k8s-1 ingress]# kubectl get svc
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
sc-nginx-svc         ClusterIP   10.100.3.0      <none>        80/TCP           63s
# 访问服务暴露的ip
[root@k8s-1 ingress]# curl 10.100.3.0

2.6 启用ingress 关联ingress controller 和service

yaml 复制代码

[root@k8s-1 ingress]# cat sc-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sc-ingress
  annotations:
    kubernets.io/ingress.class: nginx #注释 这个ingress 是关联ingress controller的
spec:
  ingressClassName: nginx  #关联ingress controller
  rules:
  - host: www.feng.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc
            port:
              number: 80
  - host: www.zhang.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc-2
            port:
              number: 80
# 执行文件
[root@k8s-1 ingress]# kubectl apply -f sc-ingress.yaml   
ingress.networking.k8s.io/sc-ingress created
# 查看效果
[root@k8s-1 ingress]# kubectl get ingress
NAME         CLASS   HOSTS                        ADDRESS                           PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com   192.168.168.146,192.168.168.147   80      56s

2.7 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则

bash 复制代码

[root@k8s-1 ingress]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-rq5cg        0/1     Completed   0          46m
ingress-nginx-admission-patch-hchjm         0/1     Completed   0          46m
ingress-nginx-controller-7cd558c647-hmzmx   1/1     Running     0          46m
ingress-nginx-controller-7cd558c647-w867n   1/1     Running     0          46m

2.8 进入ingress controller对应的pod里查看nginx.conf的配置

bash 复制代码

[root@k8s-1 ingress]# kubectl exec -n ingress-nginx -it ingress-nginx-controller-7cd558c647-hmzmx -- bash
bash-5.1$ cd /etc/nginx/
bash-5.1$ cat nginx.conf|grep zhang.com
	## start server www.zhang.com
		server_name www.zhang.com ;
	## end server www.zhang.com
bash-5.1$ cat nginx.conf|grep feng.com
	## start server www.feng.com
		server_name www.feng.com ;
	## end server www.feng.com
bash-5.1$ cat nginx.conf|grep -C3 upstream_balancer
	error_log  /var/log/nginx/error.log notice;
	
	upstream upstream_balancer {
		server 0.0.0.1:1234; # placeholder
		
		balancer_by_lua_block {

获取ingress controller对应的service暴露宿主机的端口，访问宿主机和相关端口，就可以验证ingress controller是否能进行负载均衡

bash 复制代码

[root@k8s-1 ingress]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.105.212.102   <none>        80:31407/TCP,443:32222/TCP   51m
ingress-nginx-controller-admission   ClusterIP   10.106.179.205   <none>        443/TCP                      51m

# 使用域名进行访问
[root@k8s-1 ingress]# vim /etc/hosts
192.168.168.146 www.feng.com
192.168.168.147 www.zhang.com

因为我们是基于域名做的负载均衡的配置，所有必须要在浏览器里使用域名去访问，不能使用ip地址

同时ingress controller做负载均衡的时候是基于http协议的，7层负载均衡

bash 复制代码

[root@k8s-1 ingress]# curl  www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

访问www.zhang.com出现异常，503错误，是nginx内部错误,因为没创建，创建后也可以正常访问

bash 复制代码

[root@k8s-1 ingress]# curl  www.zhang.com
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>

二、基于url的负载均衡的实现

路由规则定义

yaml 复制代码

[root@k8s-1 ingress]# cat sc-ingress-url.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-url-lb-example
  annotations:
    kubernets.io/ingress.class: nginx
spec:
  ingressClassName: nginx
  rules:
  - host: www.wen.com
    http:
      paths:
      - path: /foo
        pathType: Prefix
        backend:
          service:
            name: sc-nginx-svc-3
            port:
              number: 80
      - path: /bar
        pathType: Prefix
        backend:
          service:
            name: sc-nginx-svc-4
            port:
              number: 80
[root@k8s-1 ingress]# kubectl apply -f sc-ingress-url.yaml 
ingress.networking.k8s.io/simple-fanout-example created
[root@k8s-1 ingress]# kubectl get ingress
NAME                    CLASS   HOSTS                        ADDRESS                           PORTS   AGE
sc-ingress              nginx   www.feng.com,www.zhang.com   192.168.168.146,192.168.168.147   80      18m
simple-fanout-example   nginx   www.wen.com                  192.168.168.146,192.168.168.147   80      45s

后端服务配置

yaml 复制代码

[root@master url]# cat sc-nginx-svc-3.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sc-nginx-deploy-3
  labels:
    app: sc-nginx-feng-3
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng-3
  template:
    metadata:
      labels:
        app: sc-nginx-feng-3
    spec:
      containers:
      - name: sc-nginx-feng-3
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc-3
  labels:
    app: sc-nginx-svc-3
spec:
  selector:
    app: sc-nginx-feng-3
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80

[root@master url]# cat sc-nginx-svc-4.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sc-nginx-deploy-4
  labels:
    app: sc-nginx-feng-4
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng-4
  template:
    metadata:
      labels:
        app: sc-nginx-feng-4
    spec:
      containers:
      - name: sc-nginx-feng-4
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc-4
  labels:
    app: sc-nginx-svc-4
spec:
  selector:
    app: sc-nginx-feng-4
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80

[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-3.yaml 
deployment.apps/sc-nginx-deploy-3 created
service/sc-nginx-svc-3 created
[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-4.yaml 
deployment.apps/sc-nginx-deploy-4 created
service/sc-nginx-svc-4 created

在/etc/hosts文件里添加域名解析记录

bash 复制代码

[root@k8s-1 ingress]# cat /etc/hosts
192.168.168.146 www.wen.com
192.168.168.147 www.wen.com

测试发现不能找到页面

/usr/share/nginx/html/bar 文件夹不存在，导致404错误

进入service4 对应的一个pod里，新建bar和foo文件夹以及index.html网页文件

echo "hello,bar" >bar/index.html

再次在nfs服务器上测试，多测试几次，因为service 背后的ipvs的调度算法是轮询的

bash 复制代码

[root@nfs-server ~]# curl  www.wen.com/foo/index.html
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>
[root@nfs-server ~]# curl  www.wen.com/bar/index.html
hello,bar

查看配置的负载均衡策略

bash 复制代码

[root@k8s-1 ingress]# kubectl describe ingress simple-fanout-example
Name:             simple-fanout-example
Labels:           <none>
Namespace:        default
Address:          192.168.168.146,192.168.168.147
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host         Path  Backends
  ----         ----  --------
  www.wen.com  
               /foo   sc-nginx-svc-3:80 (<none>)
               /bar   sc-nginx-svc-4:80 (<none>)
Annotations:   kubernets.io/ingress.class: nginx
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  Sync    5h35m  nginx-ingress-controller  Scheduled for sync
  Normal  Sync    5h34m  nginx-ingress-controller  Scheduled for sync