Kubernetes HPA Behavior When Deployment Is Scaled to 0

Kubernetes HPA Behavior When Deployment Is Scaled to 0

Summary

When a Kubernetes Deployment is manually or programmatically scaled down to 0 replicas, the HorizontalPodAutoscaler (HPA) does not automatically restore it to the configured minimum replica count (minReplicas).

In this state, autoscaling is effectively inactive for that workload because there are no running pods producing metrics for HPA to evaluate. To resume autoscaling, the Deployment must first be scaled above 0 (for example, 1 replica), after which HPA control can continue.

Practical Impact

  • A service can remain unintentionally offline even though an HPA exists.
  • Teams may incorrectly assume minReplicas guarantees automatic recovery from zero.
  • Incident triage can be delayed if responders focus only on HPA configuration and not current spec.replicas.

Why This Happens

HPA adjusts replica counts for a running workload based on observed metrics and policy constraints. If the target Deployment is at 0 replicas, there are no active pods for typical resource-based signals (CPU or memory), and HPA does not bootstrap the workload back up by itself in standard configurations.

Conceptually:

  1. Deployment set to 0 replicas.
  2. No pods are running.
  3. No usable workload metrics from pods.
  4. HPA cannot compute a scaling action that revives the workload from zero.

Typical Symptoms During an Incident

  • kubectl get deploy shows 0/0 or desired 0 replicas.
  • kubectl get hpa still shows the HPA object as present.
  • Traffic fails or upstream health checks fail because no pods are available.
  • No automatic recovery occurs despite minReplicas > 0.

Fast Recovery Runbook

  1. Confirm current Deployment replica state:

    bash 复制代码
    kubectl -n <namespace> get deploy <name>
  2. Manually scale above zero:

    bash 复制代码
    kubectl -n <namespace> scale deploy/<name> --replicas=1
  3. Verify pods become Ready:

    bash 复制代码
    kubectl -n <namespace> get pods -l app=<label> -w
  4. Confirm HPA is now actively evaluating and adjusting:

    bash 复制代码
    kubectl -n <namespace> get hpa <hpa-name> -w

Preventive Controls

  • Restrict direct scale-to-zero operations on HPA-managed Deployments unless explicitly intended.
  • Add alerting for HPA-managed workloads whose desired replicas become 0.
  • Use runbook automation to restore to 1 replica when unintended zeroing is detected.
  • Document ownership and expected behavior so responders know manual wake-up may be required.

Suggested Monitoring Checks

  • Detect spec.replicas == 0 for Deployments labeled as autoscaled.
  • Detect sustained zero Ready pods for critical services.
  • Correlate with manual actions, rollout pipelines, or scripted scale events.

Decision Flow

#mermaid-svg-eE5n29JJRklbXDjw{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-eE5n29JJRklbXDjw .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-eE5n29JJRklbXDjw .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-eE5n29JJRklbXDjw .error-icon{fill:#552222;}#mermaid-svg-eE5n29JJRklbXDjw .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-eE5n29JJRklbXDjw .marker{fill:#333333;stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .marker.cross{stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-eE5n29JJRklbXDjw p{margin:0;}#mermaid-svg-eE5n29JJRklbXDjw .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label text{fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label span{color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label span p{background-color:transparent;}#mermaid-svg-eE5n29JJRklbXDjw .label text,#mermaid-svg-eE5n29JJRklbXDjw span{fill:#333;color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .node rect,#mermaid-svg-eE5n29JJRklbXDjw .node circle,#mermaid-svg-eE5n29JJRklbXDjw .node ellipse,#mermaid-svg-eE5n29JJRklbXDjw .node polygon,#mermaid-svg-eE5n29JJRklbXDjw .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .rough-node .label text,#mermaid-svg-eE5n29JJRklbXDjw .node .label text,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label,#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label{text-anchor:middle;}#mermaid-svg-eE5n29JJRklbXDjw .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .rough-node .label,#mermaid-svg-eE5n29JJRklbXDjw .node .label,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label,#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label{text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .node.clickable{cursor:pointer;}#mermaid-svg-eE5n29JJRklbXDjw .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .arrowheadPath{fill:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-eE5n29JJRklbXDjw .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-eE5n29JJRklbXDjw .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .cluster text{fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster span{color:#333;}#mermaid-svg-eE5n29JJRklbXDjw div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-eE5n29JJRklbXDjw .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw rect.text{fill:none;stroke-width:0;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape,#mermaid-svg-eE5n29JJRklbXDjw .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape p,#mermaid-svg-eE5n29JJRklbXDjw .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label rect,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-eE5n29JJRklbXDjw .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-eE5n29JJRklbXDjw :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
Deployment managed by HPA
Replicas > 0?
HPA evaluates metrics and scales within policy
No running pods for normal workload metrics
Autoscaling effectively inactive
Manually scale to at least 1 replica

Incident Note Template (Formal)

Observation: The Deployment was at 0 replicas while an HPA was configured.

Behavior: HPA did not restore the workload to minReplicas from zero.

Impact: Service availability was reduced because no pods were serving traffic.

Resolution: Deployment was manually scaled to 1 replica, allowing pods to start and HPA to resume control.

Prevention: Add guardrails and alerts to prevent or quickly detect unintended scale-to-zero on autoscaled workloads.

One-Line Summary

If an HPA-managed Deployment is scaled to 0, autoscaling does not self-revive it; scale to at least 1 replica first, then HPA can take over.

相关推荐
莫陌尛.3 小时前
StarRocks Iceberg MinIO S3 301报错完整修复文档(Docker环境)
java·docker·容器
再卷还是菜3 小时前
Kubernetes(k8s) 控制器的详细介绍与总结
笔记·学习·kubernetes
Henry-SAP3 小时前
AI芯片与具身智能双突破
人工智能·云原生·sap·erp
秋风点枝13 小时前
第一篇:认识 Prometheus —— 云原生监控的基础
云原生·prometheus
tx112087635813 小时前
K8S-pod管理与控制器管理
linux·运维·kubernetes·pod
μθημα14 小时前
Kubernetes 集群部署实操:从 Harbor 私有仓库到三节点集群搭建
云原生·容器·kubernetes
深念Y18 小时前
登录日志与管理员审计日志存储决策
前端·arm开发·后端·微服务·云原生·架构
苍狗T18 小时前
k8s 控制器管理
linux·运维·云原生·容器·kubernetes
wxchyy20 小时前
手把手教你入门云计算(二):这些技术改变了世界,你也能轻松掌握
hadoop·阿里云·docker·云原生·华为云·云计算·运维开发
名字还没想好☜20 小时前
Prometheus 告警实战:写 alerting rules、用 Alertmanager 做路由分组与抑制
运维·前端·javascript·docker·kubernetes·prometheus