k8s 部署 emqx - 技术栈

安装cert-manager

使用Helm安装

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true

如果通过helm命令安装失败，可以先下载，再按照

wget https://charts.jetstack.io/charts/cert-manager-v1.16.1.tgz

k8s原生安装

参考https://cert-manager.io/docs/installation/

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml

安装EMQX Operator

helm repo add emqx https://repos.emqx.io/charts
helm repo update
helm upgrade --install emqx-operator emqx/emqx-operator \
  --namespace emqx-operator-system \
  --create-namespace

安装后，查下安装结果

kubectl get all -n emqx-operator-system

出现下载镜像失败ImagePullBackOff的状态，说明下载不到国外的镜像，需要寻找国内镜像源。使用以下命令查询具体是哪些镜像下载失败

kubectl describe pod emqx-operator-controller-manager-6d6d5f9684-rhd7f  -n emqx-operator-system

下载镜像，并上传到k8s的各个节点后，再次执行helm安装命令

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/emqx/emqx-operator-controller:2.2.25
docker tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/emqx/emqx-operator-controller:2.2.25 emqx/emqx-operator-controller:2.2.25
docker save -o emqx.tar emqx/emqx-operator-controller:2.2.25
scp emqx.tar root@10.0.2.12:/root
ssh root@10.0.2.12
docker load -i emqx.tar

等待 EMQX Operator 就绪

kubectl wait --for=condition=Ready pods -l "control-plane=controller-manager" -n emqx-operator-system

#输出
pod/emqx-operator-controller-manager-6d6d5f9684-rhd7f condition met

安装 EMQX

apiVersion: v1
kind: Namespace
metadata:
  name: ns-emqx
  
--- 
apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:
   name: emqx
   namespace: ns-emqx
spec:
   image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/emqx/emqx:5.8.1

查看 EMQX 的 k8s service

kubectl get svc -n ns-emqx

上面这种方式部署的一个无头服务的EMQX。下面通过 LoadBalancer 访问 EMQX 集群

apiVersion: v1
kind: Namespace
metadata:
  name: ns-emqx

--- 
apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:
  name: emqx
  namespace: ns-emqx
spec:
  image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/emqx/emqx:5.8.1
  listenersServiceTemplate:
    spec:
      type: LoadBalancer
  dashboardServiceTemplate:
    spec:
      type: LoadBalancer

再次查看 EMQX 的 k8s service

kubectl get svc -n ns-emqx

结果

NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                       AGE
emqx-dashboard   NodePort    10.100.118.67    <none>        18083:31321/TCP                                               19s
emqx-headless    ClusterIP   None             <none>        4370/TCP,5369/TCP                                             40s
emqx-listeners   NodePort    10.101.198.122   <none>        8883:31407/TCP,1883:31263/TCP,8083:31433/TCP,8084:32283/TCP   19s

EMQX Operator 会创建两个 EMQX Service 资源，一个是 emqx-dashboard，一个是 emqx-listeners，分别对应 EMQX 控制台和 EMQX 监听端口。

通过 NodeIP + 端口就可以访问EMQX的控制台。这里是http://127.0.0.1:31321；用户和密码：admin/public。