前言
💡 痛点: K8s 应用怎么用 Helm 打包?Chart 怎么开发?Kustomize 怎么做环境差异化?Helm 和 Kustomize 怎么选?多环境怎么管理?
🎯 解决方案: 本文系统覆盖 K8s 部署全链路:Helm Chart 开发与模板语法、values.yaml 分层管理、Kustomize Overlay 多环境覆盖、Helm vs Kustomize 对比与选择、子 Chart 与依赖管理、Chart 发布到仓库、GitOps 部署流程、生产级最佳实践。
#mermaid-svg-NuOorJ6KygVa0m0P{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NuOorJ6KygVa0m0P .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NuOorJ6KygVa0m0P .error-icon{fill:#552222;}#mermaid-svg-NuOorJ6KygVa0m0P .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NuOorJ6KygVa0m0P .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NuOorJ6KygVa0m0P .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .marker.cross{stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NuOorJ6KygVa0m0P p{margin:0;}#mermaid-svg-NuOorJ6KygVa0m0P .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label text{fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label span{color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster-label span p{background-color:transparent;}#mermaid-svg-NuOorJ6KygVa0m0P .label text,#mermaid-svg-NuOorJ6KygVa0m0P span{fill:#333;color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .node rect,#mermaid-svg-NuOorJ6KygVa0m0P .node circle,#mermaid-svg-NuOorJ6KygVa0m0P .node ellipse,#mermaid-svg-NuOorJ6KygVa0m0P .node polygon,#mermaid-svg-NuOorJ6KygVa0m0P .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .rough-node .label text,#mermaid-svg-NuOorJ6KygVa0m0P .node .label text,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label,#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label{text-anchor:middle;}#mermaid-svg-NuOorJ6KygVa0m0P .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .rough-node .label,#mermaid-svg-NuOorJ6KygVa0m0P .node .label,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label,#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label{text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .node.clickable{cursor:pointer;}#mermaid-svg-NuOorJ6KygVa0m0P .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .arrowheadPath{fill:#333333;}#mermaid-svg-NuOorJ6KygVa0m0P .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NuOorJ6KygVa0m0P .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NuOorJ6KygVa0m0P .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster text{fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P .cluster span{color:#333;}#mermaid-svg-NuOorJ6KygVa0m0P div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NuOorJ6KygVa0m0P .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NuOorJ6KygVa0m0P rect.text{fill:none;stroke-width:0;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape p,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NuOorJ6KygVa0m0P .icon-shape .label rect,#mermaid-svg-NuOorJ6KygVa0m0P .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NuOorJ6KygVa0m0P .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NuOorJ6KygVa0m0P .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NuOorJ6KygVa0m0P :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} GitOps 部署流程
Git Repo
声明式配置
ArgoCD
自动同步
Kubernetes
集群
Kustomize 部署流程
base/
基础 YAML
overlays/
环境覆盖
kustomize build
生成最终 YAML
kubectl apply
部署到集群
Helm 部署流程
Chart 开发
templates/values
helm package
打包发布
Chart Repository
OCI/ChartMuseum
helm install
部署到集群
一、Helm Chart 开发
1.1 Chart 结构
bash
# ======== Helm Chart 标准目录结构 ========
myapp-chart/
├── Chart.yaml # Chart 元数据(名称/版本/依赖)
├── values.yaml # 默认配置值
├── values.schema.json # values JSON Schema(类型校验)
├── README.md # Chart 说明文档
├── .helmignore # 打包时排除的文件
│
├── templates/ # K8s 资源模板
│ ├── deployment.yaml # Deployment 模板
│ ├── service.yaml # Service 模板
│ ├── ingress.yaml # Ingress 模板
│ ├── configmap.yaml # ConfigMap 模板
│ ├── secret.yaml # Secret 模板(不建议存储密文)
│ ├── hpa.yaml # HorizontalPodAutoscaler
│ ├── pdb.yaml # PodDisruptionBudget
│ ├── serviceaccount.yaml # ServiceAccount
│ ├── networkpolicy.yaml # NetworkPolicy
│ ├── _helpers.tpl # 共享模板辅助函数
│ ├── NOTES.txt # 安装后提示信息
│ │
│ └── tests/ # Chart 测试
│ └── test-connection.yaml
│
├── crds/ # CustomResourceDefinitions(安装前加载)
│ └── mycrd.yaml
│
└── charts/ # 依赖子 Chart(手工管理)
│ └── redis/
│ ├── Chart.yaml
│ └── values.yaml
1.2 Chart.yaml 与 values.yaml
yaml
# ======== Chart.yaml ========
apiVersion: v2 # Helm 3 使用 apiVersion: v2
name: myapp
description: A Helm chart for myapp microservice
type: application # application 或 library
version: 1.0.0 # Chart 版本(SemVer 2)
appVersion: "2.1.0" # 应用版本
kubeVersion: ">=1.24.0-0" # 支持的 K8s 版本范围
maintainers:
- name: devops-team
email: devops@example.com
url: https://example.com
sources:
- https://github.com/myorg/myapp
home: https://myapp.example.com
icon: https://myapp.example.com/logo.png
annotations:
category: Infrastructure
licenses: Apache-2.0
# 依赖子 Chart(推荐在 Chart.yaml 中声明)
dependencies:
- name: redis
version: "18.x.x"
repository: "https://charts.bitnami.com/bitnami"
condition: redis.enabled # values.yaml 中 redis.enabled 控制是否安装
alias: redis-cache # 别名(同一 Chart 多实例)
tags:
- cache
- database
- name: postgresql
version: "15.x.x"
repository: "https://charts.bitnami.com/bitnami"
condition: postgresql.enabled
tags:
- database
- name: common
version: "2.x.x"
repository: "https://charts.bitnami.com/bitnami"
tags:
- shared
yaml
# ======== values.yaml(默认值)=======
# 全局配置
global:
imageRegistry: ghcr.io
imagePullSecrets:
- name: registry-secret
storageClass: standard
# 镜像配置
image:
registry: ghcr.io
repository: myorg/myapp
tag: "2.1.0"
digest: "" # 优先级高于 tag
pullPolicy: Always
pullSecrets: []
# 副本数
replicaCount: 2
# Pod 管理策略
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
# 资源限制
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
# 生命周期
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 10"]
# 优雅终止
terminationGracePeriodSeconds: 60
# 环境变量
extraEnvVars:
- name: LOG_LEVEL
value: info
- name: ENVIRONMENT
value: production
extraEnvVarsCM: myapp-config # 从 ConfigMap 引用
extraEnvVarsSecret: myapp-secrets # 从 Secret 引用
# 命令与参数
command: []
args: []
# 健康检查
livenessProbe:
enabled: true
httpGet:
path: /healthz/live
port: http
initialDelaySeconds: 15
periodSeconds: 20
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
enabled: true
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
startupProbe:
enabled: true
httpGet:
path: /healthz/startup
port: http
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 30 # 5*30=150s 启动超时
# Service 配置
service:
type: ClusterIP
ports:
http: 8080
grpc: 9090
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
clusterIP: "" # Headless Service(设为 None)
# Ingress 配置
ingress:
enabled: true
ingressClassName: nginx
hostname: myapp.example.com
path: /
pathType: Prefix
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rate-limit: "100"
tls: true
tlsSecret: myapp-tls
extraHosts:
- name: myapp.internal.example.com
path: /
# HPA 配置
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetCPU: 80
targetMemory: 80
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 15
# PDB 配置
pdb:
enabled: true
minAvailable: 1
# 或 maxUnavailable: "50%"
# NetworkPolicy 配置
networkPolicy:
enabled: true
allowExternal: false
ingressRules:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- port: 8080
protocol: TCP
# ServiceAccount 配置
serviceAccount:
create: true
name: ""
annotations:
iam.gke.io/gcp-service-account: myapp-sa@project.iam.gserviceaccount.com
automountServiceAccountToken: true
# 安全上下文
podSecurityContext:
enabled: true
runAsUser: 1001
runAsGroup: 1001
fsGroup: 1001
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containerSecurityContext:
enabled: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
# 节点调度
nodeSelector: {}
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-pool
operator: In
values: [production]
tolerations:
- key: dedicated
operator: Equal
value: production
effect: NoSchedule
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: myapp
# 持久化(如果需要)
persistence:
enabled: false
storageClass: standard
accessMode: ReadWriteOnce
size: 10Gi
mountPath: /data
# 初始化容器
initContainers:
- name: wait-for-db
image: busybox:1.36
command: ['sh', '-c', 'until nslookup postgresql; do echo waiting; sleep 2; done']
# Sidecar 容器
sidecars:
- name: log-collector
image: fluent/fluent-bit:2.2
volumeMounts:
- name: logs
mountPath: /var/log/app
# 额外卷
extraVolumes:
- name: config
configMap:
name: myapp-extra-config
extraVolumeMounts:
- name: config
mountPath: /etc/app/config
# 子 Chart 配置
redis:
enabled: true
architecture: standalone
auth:
enabled: true
password: ""
master:
persistence:
enabled: true
size: 8Gi
postgresql:
enabled: true
auth:
username: myapp
password: ""
database: myapp_db
primary:
persistence:
enabled: true
size: 50Gi
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 1Gi
1.3 模板语法详解
yaml
# ======== templates/_helpers.tpl(核心辅助模板)=======
{{/*
应用名称(截断 63 字符,K8s 名称限制)
*/}}
{{- define "myapp.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end -}}
{{/*
全名(Chart.Name + override,截断 63 字符)
*/}}
{{- define "myapp.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- if contains .Chart.Name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" }}
{{- end -}}
{{- end -}}
{{- end -}}
{{/*
Chart 版本标签
*/}}
{{- define "myapp.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end -}}
{{/*
通用标签(推荐所有资源都包含)
*/}}
{{- define "myapp.labels" -}}
helm.sh/chart: {{ include "myapp.chart" . }}
{{ include "myapp.selectorLabels" . }}
app.kubernetes.io/version: {{ .Values.image.tag | default .Chart.AppVersion }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
{{/*
选择器标签(Deployment/Service selector 必须匹配)
*/}}
{{- define "myapp.selectorLabels" -}}
app.kubernetes.io/name: {{ include "myapp.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}
{{/*
ServiceAccount 名称
*/}}
{{- define "myapp.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (printf "%s" (include "myapp.fullname" .)) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}
{{/*
镜像全路径
*/}}
{{- define "myapp.image" -}}
{{- $registry := .Values.image.registry | default .Values.global.imageRegistry -}}
{{- $repository := .Values.image.repository -}}
{{- $tag := .Values.image.tag | default .Chart.AppVersion -}}
{{- if .Values.image.digest }}
{{- printf "%s/%s@%s" $registry $repository .Values.image.digest }}
{{- else }}
{{- printf "%s/%s:%s" $registry $repository $tag }}
{{- end -}}
{{- end -}}
yaml
# ======== templates/deployment.yaml ========
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "myapp.fullname" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 4 }}
{{- end }}
annotations:
{{- with .Values.deploymentAnnotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
replicas: {{ .Values.replicaCount }}
# 注意:replicas 不写 selector(HPA 会修改 replicas)
{{- if .Values.updateStrategy }}
strategy:
{{- toYaml .Values.updateStrategy | nindent 4 }}
{{- end }}
selector:
matchLabels:
{{- include "myapp.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "myapp.selectorLabels" . | nindent 8 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
annotations:
# 自动滚动更新(ConfigMap/Secret 变更时触发)
checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
checksum/secrets: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
serviceAccountName: {{ include "myapp.serviceAccountName" . }}
terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
{{- with .Values.initContainers }}
initContainers:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: {{ .Chart.Name }}
image: {{ include "myapp.image" . }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
{{- if .Values.command }}
command: {{ .Values.command | toYaml | nindent 12 }}
{{- end }}
{{- if .Values.args }}
args: {{ .Values.args | toYaml | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.service.ports.http }}
protocol: TCP
{{- if .Values.service.ports.grpc }}
- name: grpc
containerPort: {{ .Values.service.ports.grpc }}
protocol: TCP
{{- end }}
env:
# 从 values 直接定义的环境变量
{{- with .Values.extraEnvVars }}
{{- toYaml . | nindent 8 }}
{{- end }}
# 从 ConfigMap 引用的环境变量
{{- if .Values.extraEnvVarsCM }}
- name: CONFIG_SOURCE
value: {{ .Values.extraEnvVarsCM }}
{{- end }}
# 从 Secret 引用的环境变量
{{- if .Values.extraEnvVarsSecret }}
- name: SECRET_SOURCE
value: {{ .Values.extraEnvVarsSecret }}
{{- end }}
# 内置环境变量
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- if .Values.livenessProbe.enabled }}
livenessProbe:
{{- toYaml .Values.livenessProbe | nindent 12 | remove "enabled" }}
{{- end }}
{{- if .Values.readinessProbe.enabled }}
readinessProbe:
{{- toYaml .Values.readinessProbe | nindent 12 | remove "enabled" }}
{{- end }}
{{- if .Values.startupProbe.enabled }}
startupProbe:
{{- toYaml .Values.startupProbe | nindent 12 | remove "enabled" }}
{{- end }}
{{- if .Values.resources }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- end }}
{{- if .Values.lifecycle }}
lifecycle:
{{- toYaml .Values.lifecycle | nindent 12 }}
{{- end }}
{{- if .Values.containerSecurityContext.enabled }}
securityContext:
{{- omit .Values.containerSecurityContext "enabled" | toYaml | nindent 12 }}
{{- end }}
volumeMounts:
{{- if .Values.persistence.enabled }}
- name: data
mountPath: {{ .Values.persistence.mountPath }}
{{- end }}
{{- with .Values.extraVolumeMounts }}
{{- toYaml . | nindent 8 }}
{{- end }}
- name: tmp
mountPath: /tmp
{{- with .Values.sidecars }}
{{- toYaml . | nindent 6 }}
{{- end }}
volumes:
{{- if .Values.persistence.enabled }}
- name: data
persistentVolumeClaim:
claimName: {{ include "myapp.fullname" . }}-data
{{- end }}
- name: tmp
emptyDir: {}
{{- with .Values.extraVolumes }}
{{- toYaml . | nindent 6 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.topologySpreadConstraints }}
topologySpreadConstraints:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if .Values.podSecurityContext.enabled }}
securityContext:
{{- omit .Values.podSecurityContext "enabled" | toYaml | nindent 8 }}
{{- end }}
yaml
# ======== templates/service.yaml ========
apiVersion: v1
kind: Service
metadata:
name: {{ include "myapp.fullname" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.service.type }}
{{- if and .Values.service.clusterIP (eq .Values.service.type "ClusterIP") }}
clusterIP: {{ .Values.service.clusterIP }}
{{- end }}
ports:
- name: http
port: {{ .Values.service.ports.http }}
targetPort: http
protocol: TCP
{{- if .Values.service.ports.grpc }}
- name: grpc
port: {{ .Values.service.ports.grpc }}
targetPort: grpc
protocol: TCP
{{- end }}
selector:
{{- include "myapp.selectorLabels" . | nindent 4 }}
yaml
# ======== templates/ingress.yaml ========
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "myapp.fullname" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.ingressClassName }}
{{- if .Values.ingress.tls }}
tls:
- hosts:
- {{ .Values.ingress.hostname }}
secretName: {{ .Values.ingress.tlsSecret }}
{{- end }}
rules:
- host: {{ .Values.ingress.hostname }}
http:
paths:
- path: {{ .Values.ingress.path }}
pathType: {{ .Values.ingress.pathType }}
backend:
service:
name: {{ include "myapp.fullname" . }}
port:
name: http
{{- range .Values.ingress.extraHosts }}
- host: {{ .name }}
http:
paths:
- path: {{ .path | default "/" }}
pathType: {{ .pathType | default "Prefix" }}
backend:
service:
name: {{ include "myapp.fullname" $ }}
port:
name: http
{{- end }}
{{- end }}
yaml
# ======== templates/configmap.yaml ========
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "myapp.fullname" . }}-config
namespace: {{ .Release.Namespace }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
data:
{{- with .Values.extraEnvVars }}
{{- range . }}
{{ .name }}: {{ .value | quote }}
{{- end }}
{{- end }}
APP_ENV: {{ .Values.env | default "production" | quote }}
LOG_FORMAT: "json"
yaml
# ======== templates/hpa.yaml ========
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "myapp.fullname" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "myapp.fullname" . }}
minReplicas: {{ .Values.autoscaling.minReplicas }}
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: {{ .Values.autoscaling.targetCPU }}
{{- if .Values.autoscaling.targetMemory }}
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: {{ .Values.autoscaling.targetMemory }}
{{- end }}
{{- if .Values.autoscaling.behavior }}
behavior:
{{- toYaml .Values.autoscaling.behavior | nindent 4 }}
{{- end }}
{{- end }}
二、Helm 多环境管理
2.1 Values 分层
yaml
# ======== values.yaml(默认值)=======
# 如上所示(基础配置)
# ======== values-production.yaml(生产环境覆盖)=======
replicaCount: 5
image:
tag: "2.1.0"
pullPolicy: IfNotPresent
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
service:
type: ClusterIP
ingress:
enabled: true
hostname: myapp.example.com
tls: true
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
autoscaling:
enabled: true
minReplicas: 5
maxReplicas: 20
targetCPU: 70
pdb:
enabled: true
minAvailable: 2
networkPolicy:
enabled: true
allowExternal: false
podSecurityContext:
runAsUser: 1001
runAsNonRoot: true
nodeSelector:
node-pool: production
tolerations:
- key: dedicated
operator: Equal
value: production
effect: NoSchedule
redis:
architecture: replication
auth:
enabled: true
master:
persistence:
size: 16Gi
replica:
replicaCount: 2
persistence:
size: 16Gi
postgresql:
primary:
persistence:
size: 100Gi
resources:
limits:
cpu: 2000m
memory: 4Gi
# ======== values-staging.yaml(测试环境覆盖)=======
replicaCount: 2
image:
tag: "2.2.0-beta" # 测试环境用最新版本
pullPolicy: Always
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
ingress:
enabled: true
hostname: myapp-staging.example.com
tls: false # 测试环境不用 TLS
autoscaling:
enabled: false # 测试环境不需要 HPA
pdb:
enabled: false
redis:
architecture: standalone
auth:
enabled: false
postgresql:
primary:
persistence:
size: 20Gi
# ======== values-development.yaml(开发环境覆盖)=======
replicaCount: 1
image:
tag: "latest"
pullPolicy: Always
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 200m
memory: 256Mi
ingress:
enabled: false
autoscaling:
enabled: false
redis:
enabled: false # 开发环境不装 Redis
postgresql:
enabled: false # 开发环境不装 PostgreSQL
livenessProbe:
enabled: false
readinessProbe:
enabled: false
2.2 Helm 命令实战
bash
# ======== Helm 命令速查 ========
# 安装
helm install myapp ./myapp-chart \
--namespace production \
--create-namespace \
-f values-production.yaml \
--set image.tag=v2.1.0 \
--set replicaCount=5 \
--timeout 5m \
--wait
# 升级
helm upgrade myapp ./myapp-chart \
--namespace production \
-f values-production.yaml \
--set image.tag=v2.2.0 \
--atomic # 失败自动回滚
# 安装或升级(推荐)
helm upgrade --install myapp ./myapp-chart \
--namespace production \
-f values-production.yaml \
--set image.tag=v2.2.0 \
--atomic
# 回滚
helm rollback myapp 1 # 回滚到 revision 1
helm rollback myapp 0 # 回滚到上一个 revision
# 查看
helm list --namespace production
helm status myapp --namespace production
helm history myapp --namespace production
helm get values myapp --namespace production
# 卸载
helm uninstall myapp --namespace production \
--keep-history # 保留历史记录
# 模板渲染(不部署)
helm template myapp ./myapp-chart \
-f values-production.yaml \
--debug
# 验证(Dry Run)
helm install --dry-run myapp ./myapp-chart \
-f values-production.yaml \
--debug
# 多值文件合并(后面的覆盖前面的)
helm upgrade --install myapp ./myapp-chart \
-f values.yaml \
-f values-production.yaml \
-f values-production-us-east.yaml
# 设置单个值
helm upgrade --install myapp ./myapp-chart \
--set image.tag=v2.1.0 \
--set service.type=LoadBalancer \
--set ingress.hostname=myapp.example.com
# 设置数组值
helm upgrade --install myapp ./myapp-chart \
--set 'extraEnvVars[0].name=LOG_LEVEL' \
--set 'extraEnvVars[0].value=debug' \
--set 'extraEnvVars[1].name=DEBUG_MODE' \
--set 'extraEnvVars[1].value=true'
# 设置 JSON/YAML 块
helm upgrade --install myapp ./myapp-chart \
--set-json 'affinity={"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-pool","operator":"In","values":["production"]}]}]}}}'
# 查看渲染差异
helm diff upgrade myapp ./myapp-chart \
-f values-production.yaml \
--set image.tag=v2.2.0 \
# 需安装 helm-diff 插件:helm plugin install https://github.com/databus23/helm-diff
# ======== Helm 插件 ========
# helm-secrets(加密 values)
helm plugin install https://github.com/jkroepke/helm-secrets
helm secrets upgrade --install myapp ./myapp-chart \
-f values-production.yaml \
-f secrets.yaml.enc
# helm-diff(渲染差异)
helm plugin install https://github.com/databus23/helm-diff
# helm-git(从 Git 安装)
helm plugin install https://github.com/aslafy-z/helm-git
三、Kustomize Overlay 实战
3.1 完整 Overlay 结构
yaml
# ======== base/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
- configmap.yaml
commonLabels:
app.kubernetes.io/name: myapp
app.kubernetes.io/part-of: myorg
configMapGenerator:
- name: myapp-config
literals:
- LOG_FORMAT=json
- METRICS_ENABLED=true
secretGenerator:
- name: myapp-secrets
literals:
- DB_PASSWORD=changeme # 不推荐,使用 SOPS/ESO
type: Opaque
yaml
# ======== overlays/production/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- hpa.yaml
- pdb.yaml
- networkpolicy.yaml
- ingress.yaml
namespace: production
namePrefix: prod-
nameSuffix: -v2
commonLabels:
app.kubernetes.io/instance: production
env: production
commonAnnotations:
note: "Production deployment - managed by ArgoCD"
images:
- name: myorg/myapp
newName: ghcr.io/myorg/myapp
newTag: v2.1.0
replicas:
- name: myapp
count: 5
patches:
# 资源限制增强
- target:
group: apps
version: v1
kind: Deployment
name: myapp
patch: |
- op: replace
path: /spec/template/spec/containers/0/resources/requests/cpu
value: "200m"
- op: replace
path: /spec/template/spec/containers/0/resources/requests/memory
value: "512Mi"
- op: replace
path: /spec/template/spec/containers/0/resources/limits/cpu
value: "1000m"
- op: replace
path: /spec/template/spec/containers/0/resources/limits/memory
value: "1Gi"
# 节点调度
- target:
kind: Deployment
patch: |
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
template:
spec:
nodeSelector:
node-pool: production
tolerations:
- key: dedicated
operator: Equal
value: production
effect: NoSchedule
# 健康检查调整
- target:
kind: Deployment
patch: |
- op: replace
path: /spec/template/spec/containers/0/livenessProbe/initialDelaySeconds
value: 30
- op: replace
path: /spec/template/spec/containers/0/readinessProbe/initialDelaySeconds
value: 10
yaml
# ======== overlays/production/hpa.yaml ========
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: prod-myapp-v2
minReplicas: 5
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
behavior:
scaleDown:
stabilizationWindowSeconds: 300
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 15
yaml
# ======== overlays/production/pdb.yaml ========
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp
spec:
minAvailable: 2
selector:
matchLabels:
app.kubernetes.io/name: myapp
env: production
yaml
# ======== overlays/production/networkpolicy.yaml ========
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: myapp-allow-frontend
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: myapp
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
yaml
# ======== overlays/staging/kustomization.yaml ========
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namespace: staging
namePrefix: staging-
commonLabels:
env: staging
images:
- name: myorg/myapp
newName: ghcr.io/myorg/myapp
newTag: v2.2.0-beta
replicas:
- name: myapp
count: 2
patches:
- target:
kind: Deployment
patch: |
- op: replace
path: /spec/template/spec/containers/0/resources/limits/cpu
value: "500m"
- op: replace
path: /spec/template/spec/containers/0/resources/limits/memory
value: "512Mi"
3.2 Kustomize 命令实战
bash
# ======== Kustomize 命令速查 ========
# 构建渲染
kustomize build overlays/production/
# 直接 apply
kubectl apply -k overlays/production/
# kubectl 内置 kustomize
kubectl apply -k overlays/production/
kubectl diff -k overlays/production/
# 查看差异
kustomize build overlays/production/ |kubectl diff -f -
# 删除
kubectl delete -k overlays/production/
# 编辑(修改后重新 apply)
kubectl edit kustomization overlays/production/
# ======== Helm + Kustomize 混合 ========
# 从 Helm Chart 生成 Kustomize base
helm template myapp ./myapp-chart \
-f values.yaml \
--output-dir ./base/
# 然后用 Kustomize overlay 覆盖
kustomize build overlays/production/
四、Helm vs Kustomize 对比
yaml
# ======== Helm vs Kustomize 全对比 ========
# | 维度 | Helm | Kustomize |
# |------------------|-------------------------------|------------------------------|
# | 核心理念 | 包管理器(打包+发布+安装) | 配置定制器(覆盖+差异化) |
# | 模板引擎 | Go template({{ .Values.xxx }})| 无模板(纯 YAML patch) |
# | 依赖管理 | Chart.yaml dependencies | 无内置依赖 |
# | 版本管理 | Chart version + AppVersion | 无版本概念(Git 管理) |
# | 发布机制 | Chart Repository/OCI Registry | 无发布机制(Git 仓库) |
# | 多环境管理 | 多个 values-xxx.yaml 文件 | overlays/ 目录结构 |
# | 回滚 | helm rollback | git revert |
# | 加密 | helm-secrets/SOPS | SOPS/ESO |
# | 生命周期钩子 | pre-install/post-install 等 | 无 |
# | 测试 | helm test | 无内置 |
# | 复杂度 | 中高(模板语法) | 低(纯 YAML) |
# | 适用场景 | 复杂应用+子 Chart+共享 | 简单覆盖+GitOps |
# ======== 选择建议 ========
# 1. 如果你是应用开发者,需要打包+发布+共享:Helm
# 2. 如果你是运维,只需要环境差异化:Kustomize
# 3. 如果两者都需要:Helm 生成 base + Kustomize overlay
# 4. 如果用 ArgoCD:两者都支持,推荐 Kustomize(GitOps 更自然)
# ======== 混合方案:Helm Chart + Kustomize Overlay ========
# 流程:
# 1. 用 Helm Chart 打包应用
# 2. 发布到 Chart Repository
# 3. 在 GitOps 仓库中用 ArgoCD Application 引用 Chart
# 4. 用 Kustomize overlay 做环境差异化
# ArgoCD Application 引用 Helm Chart:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp-production
spec:
source:
chart: myapp
repoURL: https://charts.myorg.example.com
targetRevision: 1.0.0
helm:
valueFiles:
- values-production.yaml
parameters:
- name: replicaCount
value: "5"
- name: image.tag
value: "v2.1.0"
destination:
server: https://kubernetes.default.svc
namespace: production
五、Chart 发布到仓库
5.1 OCI Registry 发布
bash
# ======== Helm Chart 发布到 OCI Registry ========
# 登录 OCI Registry
helm registry login ghcr.io \
--username myorg \
--password $GITHUB_TOKEN
# 打包 Chart
helm package ./myapp-chart
# 推送到 OCI Registry
helm push myapp-1.0.0.tgz oci://ghcr.io/myorg/charts
# 从 OCI Registry 安装
helm install myapp \
oci://ghcr.io/myorg/charts/myapp \
--version 1.0.0 \
-f values-production.yaml
# 查看远程 Chart 信息
helm show chart oci://ghcr.io/myorg/charts/myapp --version 1.0.0
helm show values oci://ghcr.io/myorg/charts/myapp --version 1.0.0
5.2 ChartMuseum 发布
bash
# ======== ChartMuseum 仓库 ========
# 启动 ChartMuseum(Docker)
docker run -d \
-p 8080:8080 \
-v /data/charts:/charts \
chartmuseum/chartmuseum:latest \
--storage=local \
--storage-local-rootdir=/charts
# 添加仓库
helm repo add myorg-chartmuseum http://localhost:8080
# 推送 Chart
helm push myapp-1.0.0.tgz myorg-chartmuseum
# 更新仓库索引
helm repo update
# 搜索
helm search repo myorg-chartmuseum/myapp
# 安装
helm install myapp myorg-chartmuseum/myapp \
--version 1.0.0 \
-f values-production.yaml
六、GitOps 部署流程
6.1 ArgoCD + Helm + Kustomize
yaml
# ======== ArgoCD Application(Helm + Kustomize 混合)=======
# 方式1:纯 Helm
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp-production
namespace: argocd
spec:
project: production
source:
chart: myapp
repoURL: https://charts.myorg.example.com
targetRevision: 1.0.0
helm:
valueFiles:
- $ref/values-production.yaml # 引用 Git 仓库中的 values
parameters:
- name: image.tag
value: v2.1.0
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
selfHeal: true
prune: true
---
# 方式2:纯 Kustomize
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp-production
namespace: argocd
spec:
project: production
source:
repoURL: https://github.com/myorg/infra-repo.git
targetRevision: main
path: overlays/production
kustomize:
images:
- ghcr.io/myorg/myapp:v2.1.0
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
selfHeal: true
prune: true
---
# 方式3:ApplicationSet(多集群自动部署)
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: myapp-all-clusters
namespace: argocd
spec:
generators:
- clusters:
selector:
matchLabels:
app: myapp
template:
metadata:
name: 'myapp-{{name}}'
spec:
project: default
source:
repoURL: https://github.com/myorg/infra-repo.git
targetRevision: main
path: 'overlays/{{metadata.labels.env}}'
kustomize:
images:
- ghcr.io/myorg/myapp:{{metadata.labels.version}}
destination:
server: '{{server}}'
namespace: 'myapp-{{metadata.labels.env}}'
syncPolicy:
automated:
selfHeal: true
prune: true
七、Checklist 总结
□ Helm Chart 开发
□ Chart.yaml(版本/依赖/元数据)
□ values.yaml(默认值 + 完整参数)
□ templates/_helpers.tpl(辅助模板)
□ templates/deployment.yaml
□ templates/service.yaml
□ templates/ingress.yaml
□ templates/configmap.yaml
□ templates/hpa.yaml
□ templates/pdb.yaml
□ values.schema.json(类型校验)
□ Helm 多环境管理
□ values-production.yaml
□ values-staging.yaml
□ values-development.yaml
□ -f 多文件合并
□ --set 单值覆盖
□ --set-json JSON 块覆盖
□ Helm 命令
□ helm install / upgrade / rollback
□ helm upgrade --install --atomic
□ helm template / diff
□ helm list / status / history
□ helm plugin install(diff/secrets/git)
□ Kustomize Overlay
□ base/ 基础 YAML
□ overlays/production/
□ overlays/staging/
□ patches(JSON6902/StrategicMerge)
□ images / replicas / commonLabels
□ configMapGenerator / secretGenerator
□ Kustomize 命令
□ kustomize build
□ kubectl apply -k
□ kubectl diff -k
□ Helm vs Kustomize
□ Helm:包管理+模板+依赖+发布
□ Kustomize:覆盖+差异化+GitOps
□ 混合:Helm base + Kustomize overlay
□ Chart 发布
□ OCI Registry(ghcr.io)
□ ChartMuseum
□ helm push / helm registry login
□ GitOps 集成
□ ArgoCD Application(Helm/Kustomize)
□ ApplicationSet(多集群)
□ 自动同步(selfHeal + prune)
总结
Helm vs Kustomize 选择矩阵:
| 场景 | 推荐 | 原因 |
|---|---|---|
| 单应用多环境部署 | Kustomize | 简单 overlay 覆盖 |
| 多应用共享依赖 | Helm | 子 Chart + 依赖管理 |
| 公开发布 Chart | Helm | Chart Repository 生态 |
| GitOps 部署 | Kustomize | 声明式 + ArgoCD 天然匹配 |
| 复杂模板渲染 | Helm | Go template 功能强大 |
| 简单差异化 | Kustomize | 无模板学习成本 |
生产级 Helm Chart 检查清单:
- ✅ 所有资源包含标准标签(helm.sh/chart / app.kubernetes.io/*)
- ✅ selector 标签与 Deployment template 标签匹配
- ✅ ConfigMap/Secret 变更触发滚动更新(checksum annotation)
- ✅ 健康检查(liveness/readiness/startup probe)
- ✅ 安全上下文(runAsNonRoot / drop ALL / readOnlyRootFilesystem)
- ✅ PDB(保证可用性)
- ✅ NetworkPolicy(限制入站流量)
- ✅ 资源限制(requests + limits)
- ✅ values.schema.json(类型校验)
- ✅ NOTES.txt(安装后提示)