Kubernetes HPA Behavior When Deployment Is Scaled to 0

Kubernetes HPA Behavior When Deployment Is Scaled to 0

Summary

When a Kubernetes Deployment is manually or programmatically scaled down to 0 replicas, the HorizontalPodAutoscaler (HPA) does not automatically restore it to the configured minimum replica count (minReplicas).

In this state, autoscaling is effectively inactive for that workload because there are no running pods producing metrics for HPA to evaluate. To resume autoscaling, the Deployment must first be scaled above 0 (for example, 1 replica), after which HPA control can continue.

Practical Impact

  • A service can remain unintentionally offline even though an HPA exists.
  • Teams may incorrectly assume minReplicas guarantees automatic recovery from zero.
  • Incident triage can be delayed if responders focus only on HPA configuration and not current spec.replicas.

Why This Happens

HPA adjusts replica counts for a running workload based on observed metrics and policy constraints. If the target Deployment is at 0 replicas, there are no active pods for typical resource-based signals (CPU or memory), and HPA does not bootstrap the workload back up by itself in standard configurations.

Conceptually:

  1. Deployment set to 0 replicas.
  2. No pods are running.
  3. No usable workload metrics from pods.
  4. HPA cannot compute a scaling action that revives the workload from zero.

Typical Symptoms During an Incident

  • kubectl get deploy shows 0/0 or desired 0 replicas.
  • kubectl get hpa still shows the HPA object as present.
  • Traffic fails or upstream health checks fail because no pods are available.
  • No automatic recovery occurs despite minReplicas > 0.

Fast Recovery Runbook

  1. Confirm current Deployment replica state:

    bash 复制代码
    kubectl -n <namespace> get deploy <name>
  2. Manually scale above zero:

    bash 复制代码
    kubectl -n <namespace> scale deploy/<name> --replicas=1
  3. Verify pods become Ready:

    bash 复制代码
    kubectl -n <namespace> get pods -l app=<label> -w
  4. Confirm HPA is now actively evaluating and adjusting:

    bash 复制代码
    kubectl -n <namespace> get hpa <hpa-name> -w

Preventive Controls

  • Restrict direct scale-to-zero operations on HPA-managed Deployments unless explicitly intended.
  • Add alerting for HPA-managed workloads whose desired replicas become 0.
  • Use runbook automation to restore to 1 replica when unintended zeroing is detected.
  • Document ownership and expected behavior so responders know manual wake-up may be required.

Suggested Monitoring Checks

  • Detect spec.replicas == 0 for Deployments labeled as autoscaled.
  • Detect sustained zero Ready pods for critical services.
  • Correlate with manual actions, rollout pipelines, or scripted scale events.

Decision Flow

#mermaid-svg-eE5n29JJRklbXDjw{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-eE5n29JJRklbXDjw .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-eE5n29JJRklbXDjw .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-eE5n29JJRklbXDjw .error-icon{fill:#552222;}#mermaid-svg-eE5n29JJRklbXDjw .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-eE5n29JJRklbXDjw .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-eE5n29JJRklbXDjw .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-eE5n29JJRklbXDjw .marker{fill:#333333;stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .marker.cross{stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-eE5n29JJRklbXDjw p{margin:0;}#mermaid-svg-eE5n29JJRklbXDjw .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label text{fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label span{color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster-label span p{background-color:transparent;}#mermaid-svg-eE5n29JJRklbXDjw .label text,#mermaid-svg-eE5n29JJRklbXDjw span{fill:#333;color:#333;}#mermaid-svg-eE5n29JJRklbXDjw .node rect,#mermaid-svg-eE5n29JJRklbXDjw .node circle,#mermaid-svg-eE5n29JJRklbXDjw .node ellipse,#mermaid-svg-eE5n29JJRklbXDjw .node polygon,#mermaid-svg-eE5n29JJRklbXDjw .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .rough-node .label text,#mermaid-svg-eE5n29JJRklbXDjw .node .label text,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label,#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label{text-anchor:middle;}#mermaid-svg-eE5n29JJRklbXDjw .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .rough-node .label,#mermaid-svg-eE5n29JJRklbXDjw .node .label,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label,#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label{text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .node.clickable{cursor:pointer;}#mermaid-svg-eE5n29JJRklbXDjw .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .arrowheadPath{fill:#333333;}#mermaid-svg-eE5n29JJRklbXDjw .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-eE5n29JJRklbXDjw .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-eE5n29JJRklbXDjw .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-eE5n29JJRklbXDjw .cluster text{fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw .cluster span{color:#333;}#mermaid-svg-eE5n29JJRklbXDjw div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-eE5n29JJRklbXDjw .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-eE5n29JJRklbXDjw rect.text{fill:none;stroke-width:0;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape,#mermaid-svg-eE5n29JJRklbXDjw .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape p,#mermaid-svg-eE5n29JJRklbXDjw .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-eE5n29JJRklbXDjw .icon-shape .label rect,#mermaid-svg-eE5n29JJRklbXDjw .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eE5n29JJRklbXDjw .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-eE5n29JJRklbXDjw .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-eE5n29JJRklbXDjw :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
Deployment managed by HPA
Replicas > 0?
HPA evaluates metrics and scales within policy
No running pods for normal workload metrics
Autoscaling effectively inactive
Manually scale to at least 1 replica

Incident Note Template (Formal)

Observation: The Deployment was at 0 replicas while an HPA was configured.

Behavior: HPA did not restore the workload to minReplicas from zero.

Impact: Service availability was reduced because no pods were serving traffic.

Resolution: Deployment was manually scaled to 1 replica, allowing pods to start and HPA to resume control.

Prevention: Add guardrails and alerts to prevent or quickly detect unintended scale-to-zero on autoscaled workloads.

One-Line Summary

If an HPA-managed Deployment is scaled to 0, autoscaling does not self-revive it; scale to at least 1 replica first, then HPA can take over.

相关推荐
蜡笔婧萱8 小时前
Docker介绍及其基础简介(一)
云原生·eureka
谢亮_vipxieliang10 小时前
Kubernetes 的“操作系统化”:从容器编排到 AI 基础设施控制平面
人工智能·平面·kubernetes
sbjdhjd13 小时前
云安全 | Docker 容器逃逸复盘(三):docker.sock 挂载与 Docker-in-Docker 风险
经验分享·网络安全·docker·云原生·容器·kubernetes·云安全
热爱Liunx的丘丘人13 小时前
1、什么是K8s
云原生·容器·kubernetes
帷幕落秋15 小时前
Docker的两种安装
运维·docker·容器
fengkai454515 小时前
十二、Redis -2
运维·数据库·redis·容器
DolitD17 小时前
3D应用多实例并发中的GPU隔离技术路径分析
容器·gru·接口隔离原则
姜鱼问生19 小时前
docker exec 排查容器问题:从日志到进程
运维·网络·数据库·docker·容器
小匠石钧知21 小时前
07_在k8s集群中安装ingress-nginx
nginx·容器·kubernetes·ingress
零基础12321 小时前
Docker 部署实战:从入门到精通
docker·容器