【vibe coding】Kubernetes + Nginx Ingress 实现云端Workspace容器分配与域名访问方案

Kubernetes + Nginx Ingress 实现云端Workspace容器分配与域名访问方案

k8s是很有趣的一个概念，应该了解一下
User Frontend API ContainerManager DNSManager CloudProvider 请求创建Workspace POST /workspaces 创建容器请求 创建容器实例 容器IP/ID 分配子域名(user123.yourdomain.com) DNS记录确认 返回容器+域名信息 返回Workspace访问URL 显示Workspace就绪(访问链接) User Frontend API ContainerManager DNSManager CloudProvider

---

Kubernetes + Nginx Ingress 实现云端Workspace容器分配与域名访问方案

1. 核心架构设计

用户 → 域名访问 (*.workspace.example.com) → Nginx Ingress → Kubernetes Service → User Pod (容器)

2. 关键组件及作用

组件	作用	推荐开源实现
Kubernetes	容器编排平台	原生K8s
Nginx Ingress Controller	路由流量到对应服务	kubernetes/ingress-nginx
Cert-Manager	自动管理TLS证书	jetstack/cert-manager
ExternalDNS	自动管理DNS记录	kubernetes-sigs/external-dns
Prometheus + Grafana	监控	prometheus-operator

3. 详细实现步骤

3.1 前置准备

# 安装helm（如未安装）
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# 添加常用仓库
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add jetstack https://charts.jetstack.io
helm repo update

3.2 安装Nginx Ingress Controller

# 安装Ingress-Nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=LoadBalancer \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"="nlb"

3.3 安装Cert-Manager（自动HTTPS）

# 安装CRDs
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.11.0/cert-manager.crds.yaml

# 安装cert-manager
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.11.0

3.4 配置ClusterIssuer（Let's Encrypt）

# cluster-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: your-email@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

kubectl apply -f cluster-issuer.yaml

3.5 部署Workspace管理服务

# workspace-controller.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: workspace-controller
spec:
  replicas: 1
  selector:
    matchLabels:
      app: workspace-controller
  template:
    metadata:
      labels:
        app: workspace-controller
    spec:
      containers:
      - name: controller
        image: your-registry/workspace-controller:latest
        env:
        - name: INGRESS_CLASS
          value: "nginx"
        - name: DOMAIN_SUFFIX
          value: "workspace.example.com"
---
apiVersion: v1
kind: Service
metadata:
  name: workspace-controller
spec:
  selector:
    app: workspace-controller
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

3.6 动态Workspace创建API示例

# workspace_controller.py (简化版)
from kubernetes import client, config
import random
import string

config.load_kube_config()

def create_workspace(user_id):
    # 生成随机ID
    workspace_id = f"ws-{user_id}-{''.join(random.choices(string.ascii_lowercase + string.digits, k=6))}"
    
    # 创建Namespace
    core_v1 = client.CoreV1Api()
    ns = client.V1Namespace(metadata=client.V1ObjectMeta(name=workspace_id))
    core_v1.create_namespace(ns)
    
    # 创建Deployment
    apps_v1 = client.AppsV1Api()
    container = client.V1Container(
        name="workspace",
        image="workspace-image:latest",
        ports=[client.V1ContainerPort(container_port=8080)]
    )
    deployment = client.V1Deployment(
        metadata=client.V1ObjectMeta(name="workspace", namespace=workspace_id),
        spec=client.V1DeploymentSpec(
            replicas=1,
            selector={"matchLabels": {"app": "workspace"}},
            template=client.V1PodTemplateSpec(
                metadata=client.V1ObjectMeta(labels={"app": "workspace"}),
                spec=client.V1PodSpec(containers=[container])
        )
    )
    apps_v1.create_namespaced_deployment(namespace=workspace_id, body=deployment)
    
    # 创建Service
    service = client.V1Service(
        metadata=client.V1ObjectMeta(name="workspace", namespace=workspace_id),
        spec=client.V1ServiceSpec(
            selector={"app": "workspace"},
            ports=[client.V1ServicePort(port=80, target_port=8080)]
        )
    )
    core_v1.create_namespaced_service(namespace=workspace_id, body=service)
    
    # 创建Ingress
    networking_v1 = client.NetworkingV1Api()
    ingress = client.V1Ingress(
        metadata=client.V1ObjectMeta(
            name="workspace",
            namespace=workspace_id,
            annotations={
                "nginx.ingress.kubernetes.io/rewrite-target": "/",
                "cert-manager.io/cluster-issuer": "letsencrypt-prod"
            }
        ),
        spec=client.V1IngressSpec(
            tls=[client.V1IngressTLS(
                hosts=[f"{workspace_id}.workspace.example.com"],
                secret_name=f"{workspace_id}-tls"
            )],
            rules=[client.V1IngressRule(
                host=f"{workspace_id}.workspace.example.com",
                http=client.V1HTTPIngressRuleValue(
                    paths=[client.V1HTTPIngressPath(
                        path="/",
                        path_type="Prefix",
                        backend=client.V1IngressBackend(
                            service=client.V1IngressServiceBackend(
                                name="workspace",
                                port=client.V1ServiceBackendPort(number=80)
                        )
                    )]
                )
            )]
        )
    )
    networking_v1.create_namespaced_ingress(namespace=workspace_id, body=ingress)
    
    return {
        "url": f"https://{workspace_id}.workspace.example.com",
        "status": "created"
    }

3.7 网络策略（多租户隔离）

# network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-inter-namespace
  namespace: {{WORKSPACE_NAMESPACE}}
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - podSelector: {}

4. 自动化运维增强

4.1 资源配额管理

# resource-quota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: workspace-quota
  namespace: {{WORKSPACE_NAMESPACE}}
spec:
  hard:
    requests.cpu: "2"
    requests.memory: 4Gi
    limits.cpu: "4"
    limits.memory: 8Gi
    pods: "5"

4.2 自动回收闲置Workspace

# idle_cleaner.py
from kubernetes import client, config
from datetime import datetime, timedelta

config.load_kube_config()

def cleanup_idle_workspaces(max_idle_hours=24):
    core_v1 = client.CoreV1Api()
    namespaces = core_v1.list_namespace(label_selector="type=workspace")
    
    for ns in namespaces.items:
        last_active = datetime.strptime(
            ns.metadata.annotations.get("last-active", "1970-01-01"),
            "%Y-%m-%dT%H:%M:%SZ"
        )
        if datetime.utcnow() - last_active > timedelta(hours=max_idle_hours):
            print(f"Deleting idle namespace: {ns.metadata.name}")
            core_v1.delete_namespace(ns.metadata.name)

5. 最佳实践建议

1. 域名管理优化：

   • 使用ExternalDNS自动管理DNS记录
   • 配置通配符证书（*.workspace.example.com）

2. 性能优化：

   # nginx-ingress优化参数
   controller:
     config:
       upstream-keepalive-connections: "100"
       upstream-keepalive-requests: "10000"
       keep-alive: "75s"

3. 安全加固：

   • 启用Ingress的WAF功能（如ModSecurity）
   • 为每个租户使用独立的Service Account

4. 监控方案：

   # 安装Prometheus Operator
   helm install prometheus prometheus-community/kube-prometheus-stack \
     --namespace monitoring \
     --create-namespace

6. 验证访问

创建Workspace后，可通过以下方式验证：

# 获取Ingress Controller外部IP
kubectl get svc -n ingress-nginx ingress-nginx-controller

# 测试访问（替换为实际域名）
curl -v https://ws-user123-abc123.workspace.example.com

此方案提供了完整的生产级实现，具备多租户隔离、自动HTTPS、资源配额管理等企业级功能，同时保持了良好的扩展性。