K8S+nginx+MYSQL+TOMCAT高可用架构企业自建网站

以下是基于 多Master高可用Kubernetes集群 的企业级部署详细步骤，涵盖 Nginx Ingress + MySQL高可用集群 + Tomcat负载均衡 的完整流程：

一、前置条件准备

1. 节点规划

Master节点：3台（高可用控制平面，需奇数台）
Worker节点：至少2台
操作系统：CentOS 7/8 或 Ubuntu 20.04+
网络要求：所有节点间网络互通，禁用防火墙/SELinux

2. 配置主机名及解析

复制代码

# 所有节点执行
sudo hostnamectl set-hostname master1  # 按实际修改为master1, master2, master3, worker1等
sudo vi /etc/hosts
# 添加以下内容（替换实际IP）： 192.168.1.101 master1 192.168.1.102 master2 192.168.1.103 master3 192.168.1.201 worker1 192.168.1.202 worker2

3. 安装依赖工具

复制代码

# 所有节点执行
sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common # Ubuntu # 或 sudo yum install -y yum-utils device-mapper-persistent-data lvm2 # CentOS

二、部署高可用Kubernetes集群

1. 安装Docker

复制代码

# 所有节点执行
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun sudo systemctl enable docker && sudo systemctl start docker

2. 安装kubeadm/kubelet/kubectl

复制代码

# 所有节点执行（以Ubuntu为例）
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt-get update sudo apt-get install -y kubelet=1.28.0-00 kubeadm=1.28.0-00 kubectl=1.28.0-00 sudo apt-mark hold kubelet kubeadm kubectl

3. 初始化第一个Master节点

复制代码

# 在master1节点执行
sudo kubeadm init \
  --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" \
  --upload-certs \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.28.0 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=192.168.0.0/16 \ --apiserver-advertise-address=192.168.1.101 # 输出中会包含加入其他Master和Worker的命令，保存备用 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

4. 加入其他Master节点

复制代码

# 在master2和master3执行（使用上一步生成的命令，形如）：
sudo kubeadm join LOAD_BALANCER_DNS:LOAD_BALANCER_PORT \
  --token xxxx \ --discovery-token-ca-cert-hash sha256:xxxx \ --control-plane \ --certificate-key xxxx

5. 加入Worker节点

复制代码

# 在所有Worker节点执行（使用kubeadm init输出的命令）：
sudo kubeadm join LOAD_BALANCER_DNS:LOAD_BALANCER_PORT --token xxxx --discovery-token-ca-cert-hash sha256:xxxx

6. 安装网络插件（Calico）

复制代码

kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml

三、配置存储（NFS示例）

1. 部署NFS Server（可选）

复制代码

# 在存储节点执行（例如192.168.1.250）
sudo apt-get install -y nfs-kernel-server  # Ubuntu sudo mkdir -p /data/nfs sudo chmod 777 /data/nfs sudo vi /etc/exports # 添加： /data/nfs *(rw,sync,no_root_squash) sudo exportfs -a sudo systemctl restart nfs-server

2. 部署NFS StorageClass

复制代码

# 使用Helm安装NFS Provisioner
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
helm install nfs-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
  --set nfs.server=192.168.1.250 \ --set nfs.path=/data/nfs \ --set storageClass.name=nfs-sc # 验证StorageClass kubectl get storageclass

四、部署高可用MySQL集群

1. 创建Secret存储密码

复制代码

kubectl create secret generic mysql-secret \
  --from-literal=root_password=yourpassword \
  --from-literal=replication_password=replpassword

2. 部署MySQL StatefulSet

复制代码

# mysql-ha.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata: name: mysql spec: serviceName: mysql replicas: 3 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:8.0 env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: root_password - name: MYSQL_REPLICATION_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: replication_password args: - "--server-id=$(expr $RANDOM % 100 + 1)" - "--gtid-mode=ON" - "--enforce-gtid-consistency=ON" - "--log-bin=mysql-bin" - "--binlog-format=ROW" - "--relay-log=mysql-relay" - "--innodb_flush_log_at_trx_commit=1" - "--sync_binlog=1" ports: - containerPort: 3306 volumeMounts: - name: mysql-data mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: mysql-data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "nfs-sc" resources: requests: storage: 20Gi

3. 部署MySQL服务

复制代码

# mysql-service.yaml
apiVersion: v1
kind: Service
metadata: name: mysql spec: ports: - port: 3306 clusterIP: None selector: app: mysql

复制代码

kubectl apply -f mysql-ha.yaml
kubectl apply -f mysql-service.yaml

五、部署Tomcat应用

1. 创建Tomcat Deployment

复制代码

# tomcat-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: name: tomcat-app spec: replicas: 3 selector: matchLabels: app: tomcat strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: tomcat spec: containers: - name: tomcat image: tomcat:9.0-jdk17 ports: - containerPort: 8080 env: - name: DATABASE_URL value: "jdbc:mysql://mysql.default.svc.cluster.local:3306/appdb?useSSL=false" resources: requests: cpu: "100m" memory: "512Mi" limits: cpu: "500m" memory: "1Gi" livenessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 30 periodSeconds: 10 readinessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 20 periodSeconds: 5

2. 创建Service

复制代码

# tomcat-service.yaml
apiVersion: v1
kind: Service
metadata: name: tomcat-service spec: selector: app: tomcat ports: - protocol: TCP port: 80 targetPort: 8080

复制代码

kubectl apply -f tomcat-deployment.yaml
kubectl apply -f tomcat-service.yaml

六、部署Nginx Ingress Controller

1. 使用Helm安装

复制代码

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --set controller.replicaCount=3 \ --set controller.service.type=LoadBalancer \ --set controller.service.externalTrafficPolicy=Local \ --set controller.nodeSelector."kubernetes\.io/os"=linux \ --set controller.admissionWebhooks.patch.nodeSelector."kubernetes\.io/os"=linux \ --set defaultBackend.nodeSelector."kubernetes\.io/os"=linux

2. 配置Ingress路由规则

复制代码

# ingress-rule.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: name: web-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: ingressClassName: nginx rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: tomcat-service port: number: 80

复制代码

kubectl apply -f ingress-rule.yaml

七、验证与维护

1. 查看集群状态

复制代码

kubectl get nodes -o wide
kubectl get pods -A -o wide
kubectl get svc,pv,pvc

2. 测试数据库连接

复制代码

kubectl exec -it mysql-0 -- mysql -uroot -p$(kubectl get secret mysql-secret -o jsonpath='{.data.root_password}' | base64 --decode) -e "CREATE DATABASE appdb;"

3. 访问测试

复制代码

# 获取Ingress外部IP
kubectl get svc ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

# 测试访问（替换实际IP） curl -H "Host: example.com" http://<INGRESS_IP>

4. 配置HPA自动扩缩

复制代码

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: name: tomcat-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: tomcat-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 80

八、架构示意图

复制代码

用户访问 -> 云厂商LB/Nginx Ingress (外部流量)
        ↓
K8S Ingress Controller (3副本)
       ↓
Tomcat Pods (HPA自动扩缩)
        ↓
MySQL Cluster (3节点StatefulSet)
        ↓
NFS/Ceph Persistent Volumes

补充建议

监控：部署Prometheus + Grafana监控集群状态
日志：使用EFK（Elasticsearch+Fluentd+Kibana）收集日志
备份：使用Velero定期备份K8S资源
安全：启用NetworkPolicy限制Pod间通信，使用Cert-Manager管理TLS证书

以上为完整的企业级高可用架构部署流程，需根据实际环境调整IP地址、存储配置和域名信息。