基于K8S的微服务:一、服务发现,负载均衡测试(附calico网络问题解决)

1、需求

使用K8S部署server1,server2两个服务,server2调用server1的/test1接口,需支持负载均衡

2、实现两个服务用于测试

pyserver1.py

python 复制代码
from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/test1', methods=['GET'])
def test1():
    return jsonify({"message": "This is Server1!"})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5001)

pyserver2.py

python 复制代码
import requests
from flask import Flask, jsonify

app = Flask(__name__)

SERVER1_URL = "http://server1-service.default.svc.cluster.local/test1"

@app.route('/test2', methods=['GET'])
def test2():
    try:
        response = requests.get(SERVER1_URL)
        response.raise_for_status()
        return jsonify(response.json()), response.status_code
    except requests.RequestException as e:
        return jsonify({"error": str(e)}), 500

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5002)

注意API地址http://server1-service .default .svc.cluster.local:80 /test1

与后续的server1-delpoyment-service.yaml的service配置对应:

3、制作docker镜像

  • export DOCKER_BUILDKIT=0 #解决某些版本如docker-ce:18.05构建镜像缓慢问题
  • 基础镜像使用阿里云私有仓库拉的一个python镜像,先从阿里云容器镜像服务web页面搜索到这个镜像再docker pull,拉取前需要docker login一下,账号密码同等web页面的,需要配置该账号的权限,都登录了那就不需配置/etc/docker/daemon.json的insecure-registries,registry-mirrors。
  • server2镜像的制作只需要"1"改为"2"即可
Dockerfile 复制代码
FROM alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/python:3.11.1

WORKDIR /app

RUN pip install --no-cache-dir requests  flask -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com

COPY ./pyserver1.py  .

CMD ["python", "pyserver1.py"]

执行构建:
docker build -f Dockerfile-server1 -t server1 .
docker build -f Dockerfile-server2 -t server2 .

4、编写服务部署的yaml文件

server1-deployment-service.yaml

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: server1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: label-server1
  template:
    metadata:
      labels:
        app: label-server1
    spec:
      containers:
      - name: server1-instance
        image: server1:latest
        imagePullPolicy: Never
        ports:
        - containerPort: 5001
---
apiVersion: v1
kind: Service
metadata:
  name: server1-service
  namespace: default
spec:
  selector:
    app: label-server1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 5001
  type: LoadBalancer

server2-deployment-service.yaml

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: server2
spec:
  replicas: 2
  selector:
    matchLabels:
      app: label-server2
  template:
    metadata:
      labels:
        app: label-server2
    spec:
      containers:
      - name: server2-instance
        image: server2:latest
        imagePullPolicy: Never
        ports:
        - containerPort: 5002
---        
apiVersion: v1
kind: Service
metadata:
  name: server2-service
  namespace: default
spec:
  selector:
    app: label-server2
  ports:
  - protocol: TCP
    port: 80
    targetPort: 5002
  type: LoadBalancer

去除master的不可调度污点让server1,server2可以部署一个实例到master:kubectl taint nodes 128master node-role.kubernetes.io/master:NoSchedule-

部署应用并暴露服务:
kubectl apply -f server1-deployment-service.yaml
kubectl apply -f server2-deployment-service.yaml

5、接口调用测试

请求server2的/test2接口,该接口调用了server1的/test1接口

看看负载均衡效果,如果不需要复杂的负载均衡策略,基于K8S实现微服务还是可以的。

附:K8S calico网络问题

在测试接口调用主要出现了两个网络问题。

我的K8S集群初始化命令
kubeadm init --apiserver-advertise-address=192.168.72.128 --image-repository registry.aliyuncs.com/google_containers

calico 直接apply的官方找的原文件。如果kubeadm init时指定了--pod-network-cidr,这时caclio的yamlCALICO_IPV4POOL_CIDR配置项也对应起来即可。我的K8S之前安装好后,过了一段时间没用,coredns组件异常或没有ready了。这时curl localhost:30309/test2会报错请求http://server1-service.default.svc.cluster.local:80/test1时域名解析不了。

sh 复制代码
[root@128aworker ~]# kubectl get all -A -o wide
NAMESPACE     NAME                                           READY   STATUS             RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
kube-system   pod/calico-kube-controllers-577f77cb5c-94cxn   1/1     Running            0          2m21s   10.244.35.1      127amaster   <none>           <none>
kube-system   pod/calico-node-89pdk                          0/1     CrashLoopBackOff   4          2m21s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/calico-node-bt4qx                          1/1     Running            0          2m21s   192.168.72.127   127amaster   <none>           <none>
kube-system   pod/coredns-6d56c8448f-kfcx9                   1/1     Running            0          6m26s   10.244.55.64     128aworker   <none>           <none>
kube-system   pod/coredns-6d56c8448f-zlbcb                   1/1     Running            0          6m26s   10.244.55.65     128aworker   <none>           <none>
kube-system   pod/etcd-128aworker                            1/1     Running            0          6m40s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-apiserver-128aworker                  1/1     Running            0          6m40s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-controller-manager-128aworker         1/1     Running            0          6m40s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-proxy-dk4mj                           1/1     Running            0          6m26s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-proxy-n8cnw                           1/1     Running            0          6m14s   192.168.72.127   127amaster   <none>           <none>
kube-system   pod/kube-scheduler-128aworker                  1/1     Running            0          6m40s   192.168.72.128   128aworker   <none>           <none>

NAMESPACE     NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE     SELECTOR
default       service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP                  6m43s   <none>
kube-system   service/kube-dns     ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   6m42s   k8s-app=kube-dns

master worker都kubeadm reset -f重置后重新建立集群coredns OK了,但caclio还没有ready

sh 复制代码
[root@128aworker ~]# kubectl get all -A -o wide
NAMESPACE     NAME                                           READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
kube-system   pod/calico-kube-controllers-577f77cb5c-94cxn   1/1     Running   0          8m29s   10.244.35.1      127amaster   <none>           <none>
kube-system   pod/calico-node-89pdk                          0/1     Running   6          8m29s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/calico-node-bt4qx                          0/1     Running   0          8m29s   192.168.72.127   127amaster   <none>           <none>
kube-system   pod/coredns-6d56c8448f-kfcx9                   1/1     Running   0          12m     10.244.55.64     128aworker   <none>           <none>
kube-system   pod/coredns-6d56c8448f-zlbcb                   1/1     Running   0          12m     10.244.55.65     128aworker   <none>           <none>
kube-system   pod/etcd-128aworker                            1/1     Running   0          12m     192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-apiserver-128aworker                  1/1     Running   0          12m     192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-controller-manager-128aworker         1/1     Running   0          12m     192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-proxy-dk4mj                           1/1     Running   0          12m     192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-proxy-n8cnw                           1/1     Running   0          12m     192.168.72.127   127amaster   <none>           <none>
kube-system   pod/kube-scheduler-128aworker                  1/1     Running   0          12m     192.168.72.128   128aworker   <none>           <none>

再次测试发现http://server1-service.default.svc.cluster.local:80/test1网络不通,logs/describe pod都找不到明显的错误解决思路,其实这是docker自定义host_elastic bridge网络污染了K8S的网络,或许高版本K8S/calico已经解决了该问题
calico issue 相关问题还比较多。。。

sh 复制代码
[root@128aworker ~]# docker network ls
NETWORK ID     NAME           DRIVER    SCOPE
b11900aaa7fb   bridge         bridge    local
d2685ac3c5a5   home_elastic   bridge    local
d00ae1cde2a2   host           host      local
47c26a5dffe2   none           null      local
[root@128aworker ~]# docker network rm d26

删除所有节点docker自定义网络并kubectl delete -f calico.yaml再重新apply即可

sh 复制代码
[root@128aworker home]# kubectl get all -A -o wide
NAMESPACE     NAME                                           READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
kube-system   pod/calico-kube-controllers-577f77cb5c-bgrxd   1/1     Running   0          49s     10.244.35.1      127amaster   <none>           <none>
kube-system   pod/calico-node-fbcxz                          1/1     Running   0          49s     192.168.72.128   128aworker   <none>           <none>
kube-system   pod/calico-node-jq2kc                          1/1     Running   0          49s     192.168.72.127   127amaster   <none>           <none>
kube-system   pod/coredns-6d56c8448f-qxfhg                   1/1     Running   0          2m10s   10.244.55.66     128aworker   <none>           <none>
kube-system   pod/coredns-6d56c8448f-zvrj8                   1/1     Running   0          2m10s   10.244.55.65     128aworker   <none>           <none>
kube-system   pod/etcd-128aworker                            1/1     Running   0          2m24s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-apiserver-128aworker                  1/1     Running   0          2m24s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-controller-manager-128aworker         1/1     Running   0          2m24s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-proxy-2w87z                           1/1     Running   0          106s    192.168.72.127   127amaster   <none>           <none>
kube-system   pod/kube-proxy-tdf7k                           1/1     Running   0          2m10s   192.168.72.128   128aworker   <none>           <none>
kube-system   pod/kube-scheduler-128aworker                  1/1     Running   0          2m24s   192.168.72.128   128aworker   <none>           <none>

NAMESPACE     NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE     SELECTOR
default       service/kubernetes   ClusterIP   10.1.0.1     <none>        443/TCP                  2m27s   <none>
kube-system   service/kube-dns     ClusterIP   10.1.0.10    <none>        53/UDP,53/TCP,9153/TCP   2m25s   k8s-app=kube-dns
相关推荐
Yvemil72 小时前
《开启微服务之旅:Spring Boot Web开发》(二)
前端·spring boot·微服务
维李设论2 小时前
Node.js的Web服务在Nacos中的实践
前端·spring cloud·微服务·eureka·nacos·node.js·express
探索云原生3 小时前
在 K8S 中创建 Pod 是如何使用到 GPU 的: nvidia device plugin 源码分析
ai·云原生·kubernetes·go·gpu
启明真纳3 小时前
elasticache备份
运维·elasticsearch·云原生·kubernetes
nangonghen5 小时前
在华为云通过operator部署Doris v2.1集群
kubernetes·华为云·doris·operator
Yvemil76 小时前
《开启微服务之旅:Spring Boot Web开发举例》(二)
前端·spring boot·微服务
会飞的土拨鼠呀6 小时前
chart文件结构
运维·云原生·kubernetes
一个儒雅随和的男子7 小时前
微服务详细教程之nacos和sentinel实战
微服务·架构·sentinel
Yvemil78 小时前
《开启微服务之旅:Spring Boot Web开发》(三)
前端·spring boot·微服务