Spring Cloud Alibaba 微服务 K8S 部署完整文档

Spring Cloud Alibaba 微服务 K8S 部署完整文档

📋 文档概述

本文档详细介绍了如何将基于 Spring Cloud Alibaba 的微服务架构完整部署到 Kubernetes 集群中，涵盖从基础环境准备到生产级部署的全流程。

---

1. 环境准备与前提条件

1.1 系统要求

# Kubernetes 集群版本
Kubernetes: v1.20+
Docker: 20.10+
Helm: v3.0+

# 集群节点配置
Master节点: 4核8G（最小）
Node节点: 根据业务负载配置（建议4核16G起）

# 存储要求
需要配置 StorageClass 支持动态存储卷

1.2 命名空间规划

# namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: sca-microservices
  labels:
    name: sca-microservices
    environment: production

应用命名空间：

kubectl apply -f namespace.yaml

---

2. 中间件部署

2.1 Nacos 集群部署

# nacos/nacos-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-config
  namespace: sca-microservices
data:
  application.properties: |
    spring.datasource.platform=mysql
    db.num=1
    db.url.0=jdbc:mysql://mysql-service:3306/nacos?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
    db.user=nacos
    db.password=nacos123
    nacos.core.auth.enabled=true
    nacos.core.auth.server.identity.key=serverIdentity
    nacos.core.auth.server.identity.value=security
    nacos.core.auth.plugin.nacos.token.secret.key=SecretKey012345678901234567890123456789012345678901234567890123456789

# nacos/nacos-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: sca-microservices
spec:
  serviceName: nacos
  replicas: 3
  selector:
    matchLabels:
      app: nacos
  template:
    metadata:
      labels:
        app: nacos
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - nacos
              topologyKey: kubernetes.io/hostname
      containers:
      - name: nacos
        image: nacos/nacos-server:v2.2.3
        ports:
        - containerPort: 8848
          name: client
        - containerPort: 9848
          name: raft
        - containerPort: 7848
          name: election
        env:
        - name: MODE
          value: cluster
        - name: PREFER_HOST_MODE
          value: hostname
        - name: SPRING_DATASOURCE_PLATFORM
          value: mysql
        - name: MYSQL_SERVICE_HOST
          value: "mysql-service"
        - name: MYSQL_SERVICE_DB_NAME
          value: "nacos"
        - name: MYSQL_SERVICE_USER
          value: "nacos"
        - name: MYSQL_SERVICE_PASSWORD
          value: "nacos123"
        - name: NACOS_SERVERS
          value: "nacos-0.nacos.sca-microservices.svc.cluster.local:8848 nacos-1.nacos.sca-microservices.svc.cluster.local:8848 nacos-2.nacos.sca-microservices.svc.cluster.local:8848"
        readinessProbe:
          httpGet:
            path: /nacos/
            port: 8848
          initialDelaySeconds: 30
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /nacos/
            port: 8848
          initialDelaySeconds: 30
          periodSeconds: 30
        resources:
          requests:
            memory: "2Gi"
            cpu: "500m"
          limits:
            memory: "4Gi"
            cpu: "2000m"
        volumeMounts:
        - name: nacos-logs
          mountPath: /home/nacos/logs
  volumeClaimTemplates:
  - metadata:
      name: nacos-logs
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "fast-ssd"
      resources:
        requests:
          storage: 20Gi

# nacos/nacos-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: nacos-service
  namespace: sca-microservices
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  ports:
  - port: 8848
    name: client
  - port: 9848
    name: raft
  - port: 7848
    name: election
  clusterIP: None
  selector:
    app: nacos

2.2 Redis 集群部署

# redis/redis-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: sca-microservices
data:
  redis.conf: |
    port 6379
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    appendonly yes
    protected-mode no

# redis/redis-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
  namespace: sca-microservices
spec:
  serviceName: redis
  replicas: 6
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7.0-alpine
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["redis-server"]
        args: ["/etc/redis/redis.conf"]
        env:
        - name: REDIS_PORT
          value: "6379"
        volumeMounts:
        - name: redis-config
          mountPath: /etc/redis
        - name: redis-data
          mountPath: /data
        resources:
          requests:
            memory: "1Gi"
            cpu: "200m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: redis-config
        configMap:
          name: redis-config
          items:
          - key: redis.conf
            path: redis.conf
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "fast-ssd"
      resources:
        requests:
          storage: 10Gi

---

3. 微服务部署配置

3.1 订单服务 (order-service)

# order-service/order-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
  namespace: sca-microservices
  labels:
    app: order-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
        prometheus.io/path: "/actuator/prometheus"
    spec:
      containers:
      - name: order-service
        image: registry.example.com/order-service:1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "k8s"
        - name: SPRING_CLOUD_NACOS_SERVER_ADDR
          value: "nacos-service:8848"
        - name: SPRING_REDIS_CLUSTER_NODES
          value: "redis-0.redis:6379,redis-1.redis:6379,redis-2.redis:6379,redis-3.redis:6379,redis-4.redis:6379,redis-5.redis:6379"
        - name: SPRING_DATASOURCE_URL
          value: "jdbc:mysql://mysql-service:3306/order_db?useUnicode=true&characterEncoding=utf-8&useSSL=false"
        - name: SPRING_DATASOURCE_USERNAME
          value: "order_user"
        - name: SPRING_DATASOURCE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 120
          periodSeconds: 10
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 5
          timeoutSeconds: 3
        volumeMounts:
        - name: application-config
          mountPath: /app/config
      volumes:
      - name: application-config
        configMap:
          name: order-service-config
      imagePullSecrets:
      - name: registry-secret
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

# order-service/order-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: order-service
  namespace: sca-microservices
  labels:
    app: order-service
spec:
  ports:
  - port: 8080
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: order-service

# order-service/order-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: order-service-hpa
  namespace: sca-microservices
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 120
      policies:
      - type: Percent
        value: 100
        periodSeconds: 60

3.2 支付服务 (payment-service)

# payment-service/payment-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-service
  namespace: sca-microservices
  labels:
    app: payment-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: payment-service
  template:
    metadata:
      labels:
        app: payment-service
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      containers:
      - name: payment-service
        image: registry.example.com/payment-service:1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "k8s"
        - name: SPRING_CLOUD_NACOS_SERVER_ADDR
          value: "nacos-service:8848"
        - name: WECHAT_PAY_MCH_ID
          valueFrom:
            secretKeyRef:
              name: wechat-pay-secret
              key: mch-id
        - name: WECHAT_PAY_API_KEY
          valueFrom:
            secretKeyRef:
              name: wechat-pay-secret
              key: api-key
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 120
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 5
        volumeMounts:
        - name: application-config
          mountPath: /app/config
      volumes:
      - name: application-config
        configMap:
          name: payment-service-config
      imagePullSecrets:
      - name: registry-secret

3.3 API网关 (api-gateway)

# gateway/gateway-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-gateway
  namespace: sca-microservices
  labels:
    app: api-gateway
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api-gateway
  template:
    metadata:
      labels:
        app: api-gateway
    spec:
      containers:
      - name: api-gateway
        image: registry.example.com/api-gateway:1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "k8s"
        - name: SPRING_CLOUD_NACOS_SERVER_ADDR
          value: "nacos-service:8848"
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 5

# gateway/gateway-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: api-gateway
  namespace: sca-microservices
  labels:
    app: api-gateway
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: api-gateway

---

4. 配置管理

4.1 ConfigMap 配置

# config/order-service-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: order-service-config
  namespace: sca-microservices
data:
  application-k8s.yml: |
    server:
      port: 8080
    spring:
      datasource:
        url: jdbc:mysql://mysql-service:3306/order_db?useUnicode=true&characterEncoding=utf-8&useSSL=false
        username: order_user
        password: ${SPRING_DATASOURCE_PASSWORD}
        hikari:
          maximum-pool-size: 20
          minimum-idle: 5
    mybatis-plus:
      configuration:
        map-underscore-to-camel-case: true
      mapper-locations: classpath:mapper/*.xml
    logging:
      level:
        com.example.order: DEBUG
      file:
        path: /app/logs

4.2 Secret 配置

# secrets/mysql-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
  namespace: sca-microservices
type: Opaque
data:
  username: b3JkZXJfdXNlcg==  # order_user
  password: TXlTUUxAMTIzNDU2  # MySQL@123456

# secrets/wechat-pay-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: wechat-pay-secret
  namespace: sca-microservices
type: Opaque
data:
  mch-id: MTIzNDU2Nzg5MA==  # 1234567890
  api-key: V0VDSEFUX1BBWV9BUElfS0VZ  # WECHAT_PAY_API_KEY

---

5. 网络与入口配置

5.1 Ingress 配置

# network/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sca-microservices-ingress
  namespace: sca-microservices
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
spec:
  tls:
  - hosts:
    - api.example.com
    secretName: tls-secret
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /order(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: order-service
            port:
              number: 8080
      - path: /payment(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: payment-service
            port:
              number: 8080
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-gateway
            port:
              number: 80

---

6. 监控与日志

6.1 ServiceMonitor (Prometheus)

# monitoring/order-service-servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: order-service-monitor
  namespace: sca-microservices
  labels:
    app: order-service
spec:
  selector:
    matchLabels:
      app: order-service
  endpoints:
  - port: http
    path: /actuator/prometheus
    interval: 30s
    scrapeTimeout: 10s

6.2 日志收集配置

# logging/fluentd-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: sca-microservices
data:
  fluent.conf: |
    <source>
      @type tail
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      tag kubernetes.*
      read_from_head true
      <parse>
        @type json
        time_format %Y-%m-%dT%H:%M:%S.%NZ
      </parse>
    </source>
    
    <filter kubernetes.**>
      @type kubernetes_metadata
    </filter>
    
    <match kubernetes.**>
      @type elasticsearch
      host elasticsearch-logging
      port 9200
      logstash_format true
      logstash_prefix kubernetes
    </match>

---

7. 部署脚本

7.1 批量部署脚本

#!/bin/bash
# deploy-all.sh

set -e

echo "开始部署 Spring Cloud Alibaba 微服务到 Kubernetes..."

# 创建命名空间
kubectl apply -f namespace.yaml

# 部署中间件
echo "部署中间件..."
kubectl apply -f nacos/
kubectl apply -f redis/
kubectl apply -f mysql/

# 等待中间件就绪
echo "等待中间件就绪..."
kubectl wait --for=condition=ready pod -l app=nacos -n sca-microservices --timeout=600s
kubectl wait --for=condition=ready pod -l app=redis -n sca-microservices --timeout=300s

# 部署配置
echo "部署配置..."
kubectl apply -f config/
kubectl apply -f secrets/

# 部署微服务
echo "部署微服务..."
kubectl apply -f order-service/
kubectl apply -f payment-service/
kubectl apply -f gateway/

# 部署网络
echo "部署网络配置..."
kubectl apply -f network/

# 部署监控
echo "部署监控配置..."
kubectl apply -f monitoring/

echo "部署完成！检查服务状态："
kubectl get all -n sca-microservices

7.2 健康检查脚本

#!/bin/bash
# health-check.sh

NAMESPACE="sca-microservices"

check_service() {
    local service=$1
    echo "检查 $service 服务状态..."
    
    # 检查Pod状态
    kubectl get pods -n $NAMESPACE -l app=$service
    
    # 检查服务端点
    endpoints=$(kubectl get endpoints $service -n $NAMESPACE -o jsonpath='{.subsets[*].addresses[*].ip}')
    if [ -z "$endpoints" ]; then
        echo "❌ $service 服务无可用端点"
        return 1
    else
        echo "✅ $service 服务端点: $endpoints"
    fi
}

# 检查所有服务
check_service "nacos"
check_service "redis"
check_service "order-service"
check_service "payment-service"
check_service "api-gateway"

echo "健康检查完成！"

---

8. 运维命令手册

8.1 常用运维命令

# 查看所有资源
kubectl get all -n sca-microservices

# 查看Pod日志
kubectl logs -f deployment/order-service -n sca-microservices

# 进入Pod调试
kubectl exec -it deployment/order-service -n sca-microservices -- /bin/sh

# 查看服务发现
kubectl get endpoints -n sca-microservices

# 水平扩缩容
kubectl scale deployment order-service --replicas=5 -n sca-microservices

# 重启部署
kubectl rollout restart deployment/order-service -n sca-microservices

# 查看部署历史
kubectl rollout history deployment/order-service -n sca-microservices

# 回滚部署
kubectl rollout undo deployment/order-service -n sca-microservices

8.2 故障排查命令

# 查看事件
kubectl get events -n sca-microservices --sort-by=.lastTimestamp

# 查看资源使用情况
kubectl top pods -n sca-microservices

# 查看服务详情
kubectl describe service order-service -n sca-microservices

# 端口转发调试
kubectl port-forward service/order-service 8080:8080 -n sca-microservices

---

9. 最佳实践建议

9.1 资源优化

• 请求/限制设置：根据实际监控数据调整资源请求和限制
• HPA配置：设置合理的扩缩容阈值和冷却时间
• 节点亲和性：关键服务配置节点亲和性保证高可用

9.2 安全实践

• 网络策略：配置NetworkPolicy限制不必要的网络访问
• RBAC：为服务账户配置最小权限原则
• Secret管理：使用外部Secret管理工具（如HashiCorp Vault）

9.3 监控告警

• 关键指标监控：API成功率、响应时间、错误率
• 业务指标监控：订单创建量、支付成功率
• 基础设施监控：CPU、内存、网络、存储使用率

---

📊 部署验证清单

部署完成后，使用以下清单验证部署状态：

• 所有Pod处于Running状态
• 服务发现正常（Nacos控制台可访问）
• 配置中心配置正确加载
• 数据库连接正常
• Redis集群状态正常
• API网关路由配置正确
• 监控数据正常采集
• 日志收集正常
• 外部访问正常（通过Ingress）
• 健康检查接口正常响应
• 业务功能测试通过

这份完整的K8S部署文档涵盖了Spring Cloud Alibaba微服务在Kubernetes环境中的全链路部署方案，可根据实际业务需求进行调整和优化。