Kubernetes 部署实战:Helm Chart 开发 + Kustomize Overlay + 多环境管理 + GitOps 部署

前言

💡 痛点: K8s 应用怎么用 Helm 打包?Chart 怎么开发?Kustomize 怎么做环境差异化?Helm 和 Kustomize 怎么选?多环境怎么管理?

🎯 解决方案: 本文系统覆盖 K8s 部署全链路:Helm Chart 开发与模板语法、values.yaml 分层管理、Kustomize Overlay 多环境覆盖、Helm vs Kustomize 对比与选择、子 Chart 与依赖管理、Chart 发布到仓库、GitOps 部署流程、生产级最佳实践。
#mermaid-svg-NuOorJ6KygVa0m0P{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NuOorJ6KygVa0m0P .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NuOorJ6KygVa0m0P .error-icon{fill:#552222;}#mermaid-svg-NuOorJ6KygVa0m0P .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NuOorJ6KygVa0m0P .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .marker.cross{stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NuOorJ6KygVa0m0P p{margin:0;}#mermaid-svg-NuOorJ6KygVa0m0P .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label text{fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label span{color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label span p{background-color:transparent;}#mermaid-svg-NuOorJ6KygVa0m0P .label text,#mermaid-svg-NuOorJ6KygVa0m0P span{fill:#333;color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .node rect,#mermaid-svg-NuOorJ6KygVa0m0P .node circle,#mermaid-svg-NuOorJ6KygVa0m0P .node ellipse,#mermaid-svg-NuOorJ6KygVa0m0P .node polygon,#mermaid-svg-NuOorJ6KygVa0m0P .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .rough-node .label text,#mermaid-svg-NuOorJ6KygVa0m0P .node .label text,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label,#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label{text-anchor:middle;}#mermaid-svg-NuOorJ6KygVa0m0P .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .rough-node .label,#mermaid-svg-NuOorJ6KygVa0m0P .node .label,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label,#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label{text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .node.clickable{cursor:pointer;}#mermaid-svg-NuOorJ6KygVa0m0P .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .arrowheadPath{fill:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NuOorJ6KygVa0m0P .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NuOorJ6KygVa0m0P .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster text{fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster span{color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NuOorJ6KygVa0m0P .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P rect.text{fill:none;stroke-width:0;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape p,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label rect,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NuOorJ6KygVa0m0P .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NuOorJ6KygVa0m0P :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} GitOps 部署流程
Git Repo

声明式配置
ArgoCD

自动同步
Kubernetes

集群
Kustomize 部署流程
base/

基础 YAML
overlays/

环境覆盖
kustomize build

生成最终 YAML
kubectl apply

部署到集群
Helm 部署流程
Chart 开发

templates/values
helm package

打包发布
Chart Repository

OCI/ChartMuseum
helm install

部署到集群


一、Helm Chart 开发

1.1 Chart 结构

bash 复制代码
# ======== Helm Chart 标准目录结构 ========
myapp-chart/
├── Chart.yaml                  # Chart 元数据(名称/版本/依赖)
├── values.yaml                 # 默认配置值
├── values.schema.json          # values JSON Schema(类型校验)
├── README.md                   # Chart 说明文档
├── .helmignore                 # 打包时排除的文件
│
├── templates/                  # K8s 资源模板
│   ├── deployment.yaml         # Deployment 模板
│   ├── service.yaml            # Service 模板
│   ├── ingress.yaml            # Ingress 模板
│   ├── configmap.yaml          # ConfigMap 模板
│   ├── secret.yaml             # Secret 模板(不建议存储密文)
│   ├── hpa.yaml                # HorizontalPodAutoscaler
│   ├── pdb.yaml                # PodDisruptionBudget
│   ├── serviceaccount.yaml     # ServiceAccount
│   ├── networkpolicy.yaml      # NetworkPolicy
│   ├── _helpers.tpl            # 共享模板辅助函数
│   ├── NOTES.txt               # 安装后提示信息
│   │
│   └── tests/                  # Chart 测试
│       └── test-connection.yaml
│
├── crds/                       # CustomResourceDefinitions(安装前加载)
│   └── mycrd.yaml
│
└── charts/                     # 依赖子 Chart(手工管理)
│   └── redis/
│       ├── Chart.yaml
│       └── values.yaml

1.2 Chart.yaml 与 values.yaml

yaml 复制代码
# ======== Chart.yaml ========
apiVersion: v2  # Helm 3 使用 apiVersion: v2
name: myapp
description: A Helm chart for myapp microservice
type: application  # application 或 library
version: 1.0.0    # Chart 版本(SemVer 2)
appVersion: "2.1.0"  # 应用版本
kubeVersion: ">=1.24.0-0"  # 支持的 K8s 版本范围
maintainers:
- name: devops-team
  email: devops@example.com
  url: https://example.com
sources:
- https://github.com/myorg/myapp
home: https://myapp.example.com
icon: https://myapp.example.com/logo.png
annotations:
  category: Infrastructure
  licenses: Apache-2.0

# 依赖子 Chart(推荐在 Chart.yaml 中声明)
dependencies:
- name: redis
  version: "18.x.x"
  repository: "https://charts.bitnami.com/bitnami"
  condition: redis.enabled  # values.yaml 中 redis.enabled 控制是否安装
  alias: redis-cache        # 别名(同一 Chart 多实例)
  tags:
  - cache
  - database

- name: postgresql
  version: "15.x.x"
  repository: "https://charts.bitnami.com/bitnami"
  condition: postgresql.enabled
  tags:
  - database

- name: common
  version: "2.x.x"
  repository: "https://charts.bitnami.com/bitnami"
  tags:
  - shared
yaml 复制代码
# ======== values.yaml(默认值)=======
# 全局配置
global:
  imageRegistry: ghcr.io
  imagePullSecrets:
  - name: registry-secret
  storageClass: standard

# 镜像配置
image:
  registry: ghcr.io
  repository: myorg/myapp
  tag: "2.1.0"
  digest: ""  # 优先级高于 tag
  pullPolicy: Always
  pullSecrets: []

# 副本数
replicaCount: 2

# Pod 管理策略
updateStrategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0

# 资源限制
resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

# 生命周期
lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 10"]

# 优雅终止
terminationGracePeriodSeconds: 60

# 环境变量
extraEnvVars:
- name: LOG_LEVEL
  value: info
- name: ENVIRONMENT
  value: production

extraEnvVarsCM: myapp-config  # 从 ConfigMap 引用
extraEnvVarsSecret: myapp-secrets  # 从 Secret 引用

# 命令与参数
command: []
args: []

# 健康检查
livenessProbe:
  enabled: true
  httpGet:
    path: /healthz/live
    port: http
  initialDelaySeconds: 15
  periodSeconds: 20
  timeoutSeconds: 5
  failureThreshold: 3

readinessProbe:
  enabled: true
  httpGet:
    path: /healthz/ready
    port: http
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

startupProbe:
  enabled: true
  httpGet:
    path: /healthz/startup
    port: http
  initialDelaySeconds: 0
  periodSeconds: 5
  failureThreshold: 30  # 5*30=150s 启动超时

# Service 配置
service:
  type: ClusterIP
  ports:
    http: 8080
    grpc: 9090
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "9090"
  clusterIP: ""  # Headless Service(设为 None)

# Ingress 配置
ingress:
  enabled: true
  ingressClassName: nginx
  hostname: myapp.example.com
  path: /
  pathType: Prefix
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/rate-limit: "100"
  tls: true
  tlsSecret: myapp-tls
  extraHosts:
  - name: myapp.internal.example.com
    path: /

# HPA 配置
autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetCPU: 80
  targetMemory: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15

# PDB 配置
pdb:
  enabled: true
  minAvailable: 1
  # 或 maxUnavailable: "50%"

# NetworkPolicy 配置
networkPolicy:
  enabled: true
  allowExternal: false
  ingressRules:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - port: 8080
      protocol: TCP

# ServiceAccount 配置
serviceAccount:
  create: true
  name: ""
  annotations:
    iam.gke.io/gcp-service-account: myapp-sa@project.iam.gserviceaccount.com
  automountServiceAccountToken: true

# 安全上下文
podSecurityContext:
  enabled: true
  runAsUser: 1001
  runAsGroup: 1001
  fsGroup: 1001
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

containerSecurityContext:
  enabled: true
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop: [ALL]

# 节点调度
nodeSelector: {}
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: node-pool
          operator: In
          values: [production]
tolerations:
- key: dedicated
  operator: Equal
  value: production
  effect: NoSchedule
topologySpreadConstraints:
- maxSkew: 1
  topologyKey: topology.kubernetes.io/zone
  whenUnsatisfiable: DoNotSchedule
  labelSelector:
    matchLabels:
      app: myapp

# 持久化(如果需要)
persistence:
  enabled: false
  storageClass: standard
  accessMode: ReadWriteOnce
  size: 10Gi
  mountPath: /data

# 初始化容器
initContainers:
- name: wait-for-db
  image: busybox:1.36
  command: ['sh', '-c', 'until nslookup postgresql; do echo waiting; sleep 2; done']

# Sidecar 容器
sidecars:
- name: log-collector
  image: fluent/fluent-bit:2.2
  volumeMounts:
  - name: logs
    mountPath: /var/log/app

# 额外卷
extraVolumes:
- name: config
  configMap:
    name: myapp-extra-config

extraVolumeMounts:
- name: config
  mountPath: /etc/app/config

# 子 Chart 配置
redis:
  enabled: true
  architecture: standalone
  auth:
    enabled: true
    password: ""
  master:
    persistence:
      enabled: true
      size: 8Gi

postgresql:
  enabled: true
  auth:
    username: myapp
    password: ""
    database: myapp_db
  primary:
    persistence:
      enabled: true
      size: 50Gi
    resources:
      requests:
        cpu: 250m
        memory: 256Mi
      limits:
        cpu: 1000m
        memory: 1Gi

1.3 模板语法详解

yaml 复制代码
# ======== templates/_helpers.tpl(核心辅助模板)=======
{{/*
应用名称(截断 63 字符,K8s 名称限制)
*/}}
{{- define "myapp.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end -}}

{{/*
全名(Chart.Name + override,截断 63 字符)
*/}}
{{- define "myapp.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- if contains .Chart.Name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" }}
{{- end -}}
{{- end -}}
{{- end -}}

{{/*
Chart 版本标签
*/}}
{{- define "myapp.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end -}}

{{/*
通用标签(推荐所有资源都包含)
*/}}
{{- define "myapp.labels" -}}
helm.sh/chart: {{ include "myapp.chart" . }}
{{ include "myapp.selectorLabels" . }}
app.kubernetes.io/version: {{ .Values.image.tag | default .Chart.AppVersion }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{/*
选择器标签(Deployment/Service selector 必须匹配)
*/}}
{{- define "myapp.selectorLabels" -}}
app.kubernetes.io/name: {{ include "myapp.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

{{/*
ServiceAccount 名称
*/}}
{{- define "myapp.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (printf "%s" (include "myapp.fullname" .)) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}

{{/*
镜像全路径
*/}}
{{- define "myapp.image" -}}
{{- $registry := .Values.image.registry | default .Values.global.imageRegistry -}}
{{- $repository := .Values.image.repository -}}
{{- $tag := .Values.image.tag | default .Chart.AppVersion -}}
{{- if .Values.image.digest }}
{{- printf "%s/%s@%s" $registry $repository .Values.image.digest }}
{{- else }}
{{- printf "%s/%s:%s" $registry $repository $tag }}
{{- end -}}
{{- end -}}
yaml 复制代码
# ======== templates/deployment.yaml ========
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "myapp.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
    {{- with .Values.podLabels }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
  annotations:
    {{- with .Values.deploymentAnnotations }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
spec:
  replicas: {{ .Values.replicaCount }}
  # 注意:replicas 不写 selector(HPA 会修改 replicas)
  {{- if .Values.updateStrategy }}
  strategy:
    {{- toYaml .Values.updateStrategy | nindent 4 }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "myapp.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "myapp.selectorLabels" . | nindent 8 }}
        {{- with .Values.podLabels }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
      annotations:
        # 自动滚动更新(ConfigMap/Secret 变更时触发)
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
        checksum/secrets: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
        {{- with .Values.podAnnotations }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
    spec:
      serviceAccountName: {{ include "myapp.serviceAccountName" . }}
      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
      {{- with .Values.initContainers }}
      initContainers:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      containers:
      - name: {{ .Chart.Name }}
        image: {{ include "myapp.image" . }}
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        {{- if .Values.command }}
        command: {{ .Values.command | toYaml | nindent 12 }}
        {{- end }}
        {{- if .Values.args }}
        args: {{ .Values.args | toYaml | nindent 12 }}
        {{- end }}
        ports:
        - name: http
          containerPort: {{ .Values.service.ports.http }}
          protocol: TCP
        {{- if .Values.service.ports.grpc }}
        - name: grpc
          containerPort: {{ .Values.service.ports.grpc }}
          protocol: TCP
        {{- end }}
        env:
        # 从 values 直接定义的环境变量
        {{- with .Values.extraEnvVars }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
        # 从 ConfigMap 引用的环境变量
        {{- if .Values.extraEnvVarsCM }}
        - name: CONFIG_SOURCE
          value: {{ .Values.extraEnvVarsCM }}
        {{- end }}
        # 从 Secret 引用的环境变量
        {{- if .Values.extraEnvVarsSecret }}
        - name: SECRET_SOURCE
          value: {{ .Values.extraEnvVarsSecret }}
        {{- end }}
        # 内置环境变量
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        {{- if .Values.livenessProbe.enabled }}
        livenessProbe:
          {{- toYaml .Values.livenessProbe | nindent 12 | remove "enabled" }}
        {{- end }}
        {{- if .Values.readinessProbe.enabled }}
        readinessProbe:
          {{- toYaml .Values.readinessProbe | nindent 12 | remove "enabled" }}
        {{- end }}
        {{- if .Values.startupProbe.enabled }}
        startupProbe:
          {{- toYaml .Values.startupProbe | nindent 12 | remove "enabled" }}
        {{- end }}
        {{- if .Values.resources }}
        resources:
          {{- toYaml .Values.resources | nindent 12 }}
        {{- end }}
        {{- if .Values.lifecycle }}
        lifecycle:
          {{- toYaml .Values.lifecycle | nindent 12 }}
        {{- end }}
        {{- if .Values.containerSecurityContext.enabled }}
        securityContext:
          {{- omit .Values.containerSecurityContext "enabled" | toYaml | nindent 12 }}
        {{- end }}
        volumeMounts:
        {{- if .Values.persistence.enabled }}
        - name: data
          mountPath: {{ .Values.persistence.mountPath }}
        {{- end }}
        {{- with .Values.extraVolumeMounts }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
        - name: tmp
          mountPath: /tmp
      {{- with .Values.sidecars }}
      {{- toYaml . | nindent 6 }}
      {{- end }}
      volumes:
      {{- if .Values.persistence.enabled }}
      - name: data
        persistentVolumeClaim:
          claimName: {{ include "myapp.fullname" . }}-data
      {{- end }}
      - name: tmp
        emptyDir: {}
      {{- with .Values.extraVolumes }}
      {{- toYaml . | nindent 6 }}
      {{- end }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.topologySpreadConstraints }}
      topologySpreadConstraints:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- if .Values.podSecurityContext.enabled }}
      securityContext:
        {{- omit .Values.podSecurityContext "enabled" | toYaml | nindent 8 }}
      {{- end }}
yaml 复制代码
# ======== templates/service.yaml ========
apiVersion: v1
kind: Service
metadata:
  name: {{ include "myapp.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
  {{- with .Values.service.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  type: {{ .Values.service.type }}
  {{- if and .Values.service.clusterIP (eq .Values.service.type "ClusterIP") }}
  clusterIP: {{ .Values.service.clusterIP }}
  {{- end }}
  ports:
  - name: http
    port: {{ .Values.service.ports.http }}
    targetPort: http
    protocol: TCP
  {{- if .Values.service.ports.grpc }}
  - name: grpc
    port: {{ .Values.service.ports.grpc }}
    targetPort: grpc
    protocol: TCP
  {{- end }}
  selector:
    {{- include "myapp.selectorLabels" . | nindent 4 }}
yaml 复制代码
# ======== templates/ingress.yaml ========
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "myapp.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
  {{- with .Values.ingress.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  ingressClassName: {{ .Values.ingress.ingressClassName }}
  {{- if .Values.ingress.tls }}
  tls:
  - hosts:
    - {{ .Values.ingress.hostname }}
    secretName: {{ .Values.ingress.tlsSecret }}
  {{- end }}
  rules:
  - host: {{ .Values.ingress.hostname }}
    http:
      paths:
      - path: {{ .Values.ingress.path }}
        pathType: {{ .Values.ingress.pathType }}
        backend:
          service:
            name: {{ include "myapp.fullname" . }}
            port:
              name: http
  {{- range .Values.ingress.extraHosts }}
  - host: {{ .name }}
    http:
      paths:
      - path: {{ .path | default "/" }}
        pathType: {{ .pathType | default "Prefix" }}
        backend:
          service:
            name: {{ include "myapp.fullname" $ }}
            port:
              name: http
  {{- end }}
{{- end }}
yaml 复制代码
# ======== templates/configmap.yaml ========
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "myapp.fullname" . }}-config
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
data:
  {{- with .Values.extraEnvVars }}
  {{- range . }}
  {{ .name }}: {{ .value | quote }}
  {{- end }}
  {{- end }}
  APP_ENV: {{ .Values.env | default "production" | quote }}
  LOG_FORMAT: "json"
yaml 复制代码
# ======== templates/hpa.yaml ========
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "myapp.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "myapp.fullname" . }}
  minReplicas: {{ .Values.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: {{ .Values.autoscaling.targetCPU }}
  {{- if .Values.autoscaling.targetMemory }}
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: {{ .Values.autoscaling.targetMemory }}
  {{- end }}
  {{- if .Values.autoscaling.behavior }}
  behavior:
    {{- toYaml .Values.autoscaling.behavior | nindent 4 }}
  {{- end }}
{{- end }}

二、Helm 多环境管理

2.1 Values 分层

yaml 复制代码
# ======== values.yaml(默认值)=======
# 如上所示(基础配置)

# ======== values-production.yaml(生产环境覆盖)=======
replicaCount: 5

image:
  tag: "2.1.0"
  pullPolicy: IfNotPresent

resources:
  requests:
    cpu: 200m
    memory: 512Mi
  limits:
    cpu: 1000m
    memory: 1Gi

service:
  type: ClusterIP

ingress:
  enabled: true
  hostname: myapp.example.com
  tls: true
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod

autoscaling:
  enabled: true
  minReplicas: 5
  maxReplicas: 20
  targetCPU: 70

pdb:
  enabled: true
  minAvailable: 2

networkPolicy:
  enabled: true
  allowExternal: false

podSecurityContext:
  runAsUser: 1001
  runAsNonRoot: true

nodeSelector:
  node-pool: production

tolerations:
- key: dedicated
  operator: Equal
  value: production
  effect: NoSchedule

redis:
  architecture: replication
  auth:
    enabled: true
  master:
    persistence:
      size: 16Gi
  replica:
    replicaCount: 2
    persistence:
      size: 16Gi

postgresql:
  primary:
    persistence:
      size: 100Gi
    resources:
      limits:
        cpu: 2000m
        memory: 4Gi

# ======== values-staging.yaml(测试环境覆盖)=======
replicaCount: 2

image:
  tag: "2.2.0-beta"  # 测试环境用最新版本
  pullPolicy: Always

resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

ingress:
  enabled: true
  hostname: myapp-staging.example.com
  tls: false  # 测试环境不用 TLS

autoscaling:
  enabled: false  # 测试环境不需要 HPA

pdb:
  enabled: false

redis:
  architecture: standalone
  auth:
    enabled: false

postgresql:
  primary:
    persistence:
      size: 20Gi

# ======== values-development.yaml(开发环境覆盖)=======
replicaCount: 1

image:
  tag: "latest"
  pullPolicy: Always

resources:
  requests:
    cpu: 50m
    memory: 128Mi
  limits:
    cpu: 200m
    memory: 256Mi

ingress:
  enabled: false

autoscaling:
  enabled: false

redis:
  enabled: false  # 开发环境不装 Redis

postgresql:
  enabled: false  # 开发环境不装 PostgreSQL

livenessProbe:
  enabled: false

readinessProbe:
  enabled: false

2.2 Helm 命令实战

bash 复制代码
# ======== Helm 命令速查 ========

# 安装
helm install myapp ./myapp-chart \
  --namespace production \
  --create-namespace \
  -f values-production.yaml \
  --set image.tag=v2.1.0 \
  --set replicaCount=5 \
  --timeout 5m \
  --wait

# 升级
helm upgrade myapp ./myapp-chart \
  --namespace production \
  -f values-production.yaml \
  --set image.tag=v2.2.0 \
  --atomic  # 失败自动回滚

# 安装或升级(推荐)
helm upgrade --install myapp ./myapp-chart \
  --namespace production \
  -f values-production.yaml \
  --set image.tag=v2.2.0 \
  --atomic

# 回滚
helm rollback myapp 1  # 回滚到 revision 1
helm rollback myapp 0  # 回滚到上一个 revision

# 查看
helm list --namespace production
helm status myapp --namespace production
helm history myapp --namespace production
helm get values myapp --namespace production

# 卸载
helm uninstall myapp --namespace production \
  --keep-history  # 保留历史记录

# 模板渲染(不部署)
helm template myapp ./myapp-chart \
  -f values-production.yaml \
  --debug

# 验证(Dry Run)
helm install --dry-run myapp ./myapp-chart \
  -f values-production.yaml \
  --debug

# 多值文件合并(后面的覆盖前面的)
helm upgrade --install myapp ./myapp-chart \
  -f values.yaml \
  -f values-production.yaml \
  -f values-production-us-east.yaml

# 设置单个值
helm upgrade --install myapp ./myapp-chart \
  --set image.tag=v2.1.0 \
  --set service.type=LoadBalancer \
  --set ingress.hostname=myapp.example.com

# 设置数组值
helm upgrade --install myapp ./myapp-chart \
  --set 'extraEnvVars[0].name=LOG_LEVEL' \
  --set 'extraEnvVars[0].value=debug' \
  --set 'extraEnvVars[1].name=DEBUG_MODE' \
  --set 'extraEnvVars[1].value=true'

# 设置 JSON/YAML 块
helm upgrade --install myapp ./myapp-chart \
  --set-json 'affinity={"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-pool","operator":"In","values":["production"]}]}]}}}'

# 查看渲染差异
helm diff upgrade myapp ./myapp-chart \
  -f values-production.yaml \
  --set image.tag=v2.2.0 \
  # 需安装 helm-diff 插件:helm plugin install https://github.com/databus23/helm-diff

# ======== Helm 插件 ========
# helm-secrets(加密 values)
helm plugin install https://github.com/jkroepke/helm-secrets
helm secrets upgrade --install myapp ./myapp-chart \
  -f values-production.yaml \
  -f secrets.yaml.enc

# helm-diff(渲染差异)
helm plugin install https://github.com/databus23/helm-diff

# helm-git(从 Git 安装)
helm plugin install https://github.com/aslafy-z/helm-git

三、Kustomize Overlay 实战

3.1 完整 Overlay 结构

yaml 复制代码
# ======== base/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml
- service.yaml
- configmap.yaml

commonLabels:
  app.kubernetes.io/name: myapp
  app.kubernetes.io/part-of: myorg

configMapGenerator:
- name: myapp-config
  literals:
  - LOG_FORMAT=json
  - METRICS_ENABLED=true

secretGenerator:
- name: myapp-secrets
  literals:
  - DB_PASSWORD=changeme  # 不推荐,使用 SOPS/ESO
  type: Opaque
yaml 复制代码
# ======== overlays/production/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../base
- hpa.yaml
- pdb.yaml
- networkpolicy.yaml
- ingress.yaml

namespace: production

namePrefix: prod-
nameSuffix: -v2

commonLabels:
  app.kubernetes.io/instance: production
  env: production

commonAnnotations:
  note: "Production deployment - managed by ArgoCD"

images:
- name: myorg/myapp
  newName: ghcr.io/myorg/myapp
  newTag: v2.1.0

replicas:
- name: myapp
  count: 5

patches:
# 资源限制增强
- target:
    group: apps
    version: v1
    kind: Deployment
    name: myapp
  patch: |
    - op: replace
      path: /spec/template/spec/containers/0/resources/requests/cpu
      value: "200m"
    - op: replace
      path: /spec/template/spec/containers/0/resources/requests/memory
      value: "512Mi"
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/cpu
      value: "1000m"
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/memory
      value: "1Gi"

# 节点调度
- target:
    kind: Deployment
  patch: |
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: myapp
    spec:
      template:
        spec:
          nodeSelector:
            node-pool: production
          tolerations:
          - key: dedicated
            operator: Equal
            value: production
            effect: NoSchedule

# 健康检查调整
- target:
    kind: Deployment
  patch: |
    - op: replace
      path: /spec/template/spec/containers/0/livenessProbe/initialDelaySeconds
      value: 30
    - op: replace
      path: /spec/template/spec/containers/0/readinessProbe/initialDelaySeconds
      value: 10
yaml 复制代码
# ======== overlays/production/hpa.yaml ========
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: prod-myapp-v2
  minReplicas: 5
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
yaml 复制代码
# ======== overlays/production/pdb.yaml ========
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp
      env: production
yaml 复制代码
# ======== overlays/production/networkpolicy.yaml ========
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: myapp-allow-frontend
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: myapp
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080
yaml 复制代码
# ======== overlays/staging/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../base

namespace: staging

namePrefix: staging-

commonLabels:
  env: staging

images:
- name: myorg/myapp
  newName: ghcr.io/myorg/myapp
  newTag: v2.2.0-beta

replicas:
- name: myapp
  count: 2

patches:
- target:
    kind: Deployment
  patch: |
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/cpu
      value: "500m"
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/memory
      value: "512Mi"

3.2 Kustomize 命令实战

bash 复制代码
# ======== Kustomize 命令速查 ========

# 构建渲染
kustomize build overlays/production/

# 直接 apply
kubectl apply -k overlays/production/

# kubectl 内置 kustomize
kubectl apply -k overlays/production/
kubectl diff -k overlays/production/

# 查看差异
kustomize build overlays/production/ |kubectl diff -f -

# 删除
kubectl delete -k overlays/production/

# 编辑(修改后重新 apply)
kubectl edit kustomization overlays/production/

# ======== Helm + Kustomize 混合 ========
# 从 Helm Chart 生成 Kustomize base
helm template myapp ./myapp-chart \
  -f values.yaml \
  --output-dir ./base/

# 然后用 Kustomize overlay 覆盖
kustomize build overlays/production/

四、Helm vs Kustomize 对比

yaml 复制代码
# ======== Helm vs Kustomize 全对比 ========

# | 维度             | Helm                          | Kustomize                    |
# |------------------|-------------------------------|------------------------------|
# | 核心理念         | 包管理器(打包+发布+安装)      | 配置定制器(覆盖+差异化)     |
# | 模板引擎         | Go template({{ .Values.xxx }})| 无模板(纯 YAML patch)      |
# | 依赖管理         | Chart.yaml dependencies        | 无内置依赖                   |
# | 版本管理         | Chart version + AppVersion     | 无版本概念(Git 管理)       |
# | 发布机制         | Chart Repository/OCI Registry  | 无发布机制(Git 仓库)       |
# | 多环境管理       | 多个 values-xxx.yaml 文件      | overlays/ 目录结构           |
# | 回滚             | helm rollback                  | git revert                  |
# | 加密             | helm-secrets/SOPS              | SOPS/ESO                    |
# | 生命周期钩子     | pre-install/post-install 等    | 无                          |
# | 测试             | helm test                      | 无内置                      |
# | 复杂度           | 中高(模板语法)                | 低(纯 YAML)                |
# | 适用场景         | 复杂应用+子 Chart+共享         | 简单覆盖+GitOps              |

# ======== 选择建议 ========
# 1. 如果你是应用开发者,需要打包+发布+共享:Helm
# 2. 如果你是运维,只需要环境差异化:Kustomize
# 3. 如果两者都需要:Helm 生成 base + Kustomize overlay
# 4. 如果用 ArgoCD:两者都支持,推荐 Kustomize(GitOps 更自然)

# ======== 混合方案:Helm Chart + Kustomize Overlay ========
# 流程:
# 1. 用 Helm Chart 打包应用
# 2. 发布到 Chart Repository
# 3. 在 GitOps 仓库中用 ArgoCD Application 引用 Chart
# 4. 用 Kustomize overlay 做环境差异化

# ArgoCD Application 引用 Helm Chart:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-production
spec:
  source:
    chart: myapp
    repoURL: https://charts.myorg.example.com
    targetRevision: 1.0.0
    helm:
      valueFiles:
      - values-production.yaml
      parameters:
      - name: replicaCount
        value: "5"
      - name: image.tag
        value: "v2.1.0"
  destination:
    server: https://kubernetes.default.svc
    namespace: production

五、Chart 发布到仓库

5.1 OCI Registry 发布

bash 复制代码
# ======== Helm Chart 发布到 OCI Registry ========

# 登录 OCI Registry
helm registry login ghcr.io \
  --username myorg \
  --password $GITHUB_TOKEN

# 打包 Chart
helm package ./myapp-chart

# 推送到 OCI Registry
helm push myapp-1.0.0.tgz oci://ghcr.io/myorg/charts

# 从 OCI Registry 安装
helm install myapp \
  oci://ghcr.io/myorg/charts/myapp \
  --version 1.0.0 \
  -f values-production.yaml

# 查看远程 Chart 信息
helm show chart oci://ghcr.io/myorg/charts/myapp --version 1.0.0
helm show values oci://ghcr.io/myorg/charts/myapp --version 1.0.0

5.2 ChartMuseum 发布

bash 复制代码
# ======== ChartMuseum 仓库 ========

# 启动 ChartMuseum(Docker)
docker run -d \
  -p 8080:8080 \
  -v /data/charts:/charts \
  chartmuseum/chartmuseum:latest \
  --storage=local \
  --storage-local-rootdir=/charts

# 添加仓库
helm repo add myorg-chartmuseum http://localhost:8080

# 推送 Chart
helm push myapp-1.0.0.tgz myorg-chartmuseum

# 更新仓库索引
helm repo update

# 搜索
helm search repo myorg-chartmuseum/myapp

# 安装
helm install myapp myorg-chartmuseum/myapp \
  --version 1.0.0 \
  -f values-production.yaml

六、GitOps 部署流程

6.1 ArgoCD + Helm + Kustomize

yaml 复制代码
# ======== ArgoCD Application(Helm + Kustomize 混合)=======

# 方式1:纯 Helm
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-production
  namespace: argocd
spec:
  project: production
  source:
    chart: myapp
    repoURL: https://charts.myorg.example.com
    targetRevision: 1.0.0
    helm:
      valueFiles:
      - $ref/values-production.yaml  # 引用 Git 仓库中的 values
      parameters:
      - name: image.tag
        value: v2.1.0
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      selfHeal: true
      prune: true

---
# 方式2:纯 Kustomize
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-production
  namespace: argocd
spec:
  project: production
  source:
    repoURL: https://github.com/myorg/infra-repo.git
    targetRevision: main
    path: overlays/production
    kustomize:
      images:
      - ghcr.io/myorg/myapp:v2.1.0
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      selfHeal: true
      prune: true

---
# 方式3:ApplicationSet(多集群自动部署)
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapp-all-clusters
  namespace: argocd
spec:
  generators:
  - clusters:
      selector:
        matchLabels:
          app: myapp
  template:
    metadata:
      name: 'myapp-{{name}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/myorg/infra-repo.git
        targetRevision: main
        path: 'overlays/{{metadata.labels.env}}'
        kustomize:
          images:
          - ghcr.io/myorg/myapp:{{metadata.labels.version}}
      destination:
        server: '{{server}}'
        namespace: 'myapp-{{metadata.labels.env}}'
      syncPolicy:
        automated:
          selfHeal: true
          prune: true

七、Checklist 总结

复制代码
□ Helm Chart 开发
  □ Chart.yaml(版本/依赖/元数据)
  □ values.yaml(默认值 + 完整参数)
  □ templates/_helpers.tpl(辅助模板)
  □ templates/deployment.yaml
  □ templates/service.yaml
  □ templates/ingress.yaml
  □ templates/configmap.yaml
  □ templates/hpa.yaml
  □ templates/pdb.yaml
  □ values.schema.json(类型校验)

□ Helm 多环境管理
  □ values-production.yaml
  □ values-staging.yaml
  □ values-development.yaml
  □ -f 多文件合并
  □ --set 单值覆盖
  □ --set-json JSON 块覆盖

□ Helm 命令
  □ helm install / upgrade / rollback
  □ helm upgrade --install --atomic
  □ helm template / diff
  □ helm list / status / history
  □ helm plugin install(diff/secrets/git)

□ Kustomize Overlay
  □ base/ 基础 YAML
  □ overlays/production/
  □ overlays/staging/
  □ patches(JSON6902/StrategicMerge)
  □ images / replicas / commonLabels
  □ configMapGenerator / secretGenerator

□ Kustomize 命令
  □ kustomize build
  □ kubectl apply -k
  □ kubectl diff -k

□ Helm vs Kustomize
  □ Helm:包管理+模板+依赖+发布
  □ Kustomize:覆盖+差异化+GitOps
  □ 混合:Helm base + Kustomize overlay

□ Chart 发布
  □ OCI Registry(ghcr.io)
  □ ChartMuseum
  □ helm push / helm registry login

□ GitOps 集成
  □ ArgoCD Application(Helm/Kustomize)
  □ ApplicationSet(多集群)
  □ 自动同步(selfHeal + prune)

总结

Helm vs Kustomize 选择矩阵:

场景 推荐 原因
单应用多环境部署 Kustomize 简单 overlay 覆盖
多应用共享依赖 Helm 子 Chart + 依赖管理
公开发布 Chart Helm Chart Repository 生态
GitOps 部署 Kustomize 声明式 + ArgoCD 天然匹配
复杂模板渲染 Helm Go template 功能强大
简单差异化 Kustomize 无模板学习成本

生产级 Helm Chart 检查清单:

  1. ✅ 所有资源包含标准标签(helm.sh/chart / app.kubernetes.io/*)
  2. ✅ selector 标签与 Deployment template 标签匹配
  3. ✅ ConfigMap/Secret 变更触发滚动更新(checksum annotation)
  4. ✅ 健康检查(liveness/readiness/startup probe)
  5. ✅ 安全上下文(runAsNonRoot / drop ALL / readOnlyRootFilesystem)
  6. ✅ PDB(保证可用性)
  7. ✅ NetworkPolicy(限制入站流量)
  8. ✅ 资源限制(requests + limits)
  9. ✅ values.schema.json(类型校验)
  10. ✅ NOTES.txt(安装后提示)