Kubernetes 调度器完全指南:从原理到生产实战

Kubernetes 调度器完全指南:从原理到生产实战

本文深入剖析 Kubernetes 调度器(Scheduler)的核心技术,涵盖调度器架构设计、调度流程、调度策略、节点亲和性、污点容忍、拓扑分布,以及生产环境最佳实践。掌握调度器是优化集群资源利用率和应用高可用的关键。


目录

  1. 调度器概述
  2. 调度流程详解
  3. 节点选择策略
  4. 亲和性与反亲和性
  5. 污点与容忍
  6. 拓扑分布约束
  7. 实战:生产级调度配置
  8. 常用命令速查
  9. 最佳实践与避坑指南

一、调度器概述

1.1 什么是调度器?

调度器(Scheduler)是 Kubernetes 控制平面的核心组件,负责将 Pod 分配到合适的节点上运行。
#mermaid-svg-2dFNcAwg3SJQOnza{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-2dFNcAwg3SJQOnza .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-2dFNcAwg3SJQOnza .error-icon{fill:#552222;}#mermaid-svg-2dFNcAwg3SJQOnza .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-2dFNcAwg3SJQOnza .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-2dFNcAwg3SJQOnza .marker{fill:#333333;stroke:#333333;}#mermaid-svg-2dFNcAwg3SJQOnza .marker.cross{stroke:#333333;}#mermaid-svg-2dFNcAwg3SJQOnza svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-2dFNcAwg3SJQOnza p{margin:0;}#mermaid-svg-2dFNcAwg3SJQOnza .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-2dFNcAwg3SJQOnza .cluster-label text{fill:#333;}#mermaid-svg-2dFNcAwg3SJQOnza .cluster-label span{color:#333;}#mermaid-svg-2dFNcAwg3SJQOnza .cluster-label span p{background-color:transparent;}#mermaid-svg-2dFNcAwg3SJQOnza .label text,#mermaid-svg-2dFNcAwg3SJQOnza span{fill:#333;color:#333;}#mermaid-svg-2dFNcAwg3SJQOnza .node rect,#mermaid-svg-2dFNcAwg3SJQOnza .node circle,#mermaid-svg-2dFNcAwg3SJQOnza .node ellipse,#mermaid-svg-2dFNcAwg3SJQOnza .node polygon,#mermaid-svg-2dFNcAwg3SJQOnza .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-2dFNcAwg3SJQOnza .rough-node .label text,#mermaid-svg-2dFNcAwg3SJQOnza .node .label text,#mermaid-svg-2dFNcAwg3SJQOnza .image-shape .label,#mermaid-svg-2dFNcAwg3SJQOnza .icon-shape .label{text-anchor:middle;}#mermaid-svg-2dFNcAwg3SJQOnza .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-2dFNcAwg3SJQOnza .rough-node .label,#mermaid-svg-2dFNcAwg3SJQOnza .node .label,#mermaid-svg-2dFNcAwg3SJQOnza .image-shape .label,#mermaid-svg-2dFNcAwg3SJQOnza .icon-shape .label{text-align:center;}#mermaid-svg-2dFNcAwg3SJQOnza .node.clickable{cursor:pointer;}#mermaid-svg-2dFNcAwg3SJQOnza .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-2dFNcAwg3SJQOnza .arrowheadPath{fill:#333333;}#mermaid-svg-2dFNcAwg3SJQOnza .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-2dFNcAwg3SJQOnza .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-2dFNcAwg3SJQOnza .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2dFNcAwg3SJQOnza .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-2dFNcAwg3SJQOnza .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2dFNcAwg3SJQOnza .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-2dFNcAwg3SJQOnza .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-2dFNcAwg3SJQOnza .cluster text{fill:#333;}#mermaid-svg-2dFNcAwg3SJQOnza .cluster span{color:#333;}#mermaid-svg-2dFNcAwg3SJQOnza div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-2dFNcAwg3SJQOnza .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-2dFNcAwg3SJQOnza rect.text{fill:none;stroke-width:0;}#mermaid-svg-2dFNcAwg3SJQOnza .icon-shape,#mermaid-svg-2dFNcAwg3SJQOnza .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2dFNcAwg3SJQOnza .icon-shape p,#mermaid-svg-2dFNcAwg3SJQOnza .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-2dFNcAwg3SJQOnza .icon-shape .label rect,#mermaid-svg-2dFNcAwg3SJQOnza .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2dFNcAwg3SJQOnza .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-2dFNcAwg3SJQOnza .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-2dFNcAwg3SJQOnza :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Pod 创建
调度器
过滤节点
打分排序
选择最优节点
绑定 Pod

1.2 调度器职责

职责 说明
资源匹配 确保节点有足够资源
策略应用 应用亲和性、污点等策略
负载均衡 尽量均匀分布 Pod
高可用 避免单点故障

1.3 调度器类型

类型 说明 适用场景
默认调度器 kube-scheduler 大多数场景
自定义调度器 自定义逻辑 特殊需求
多调度器 多个调度器共存 混合场景
yaml 复制代码
# 使用自定义调度器
apiVersion: v1
kind: Pod
metadata:
  name: custom-scheduler-pod
spec:
  schedulerName: my-scheduler  # 指定调度器
  containers:
  - name: app
    image: nginx

二、调度流程详解

2.1 两阶段调度

#mermaid-svg-Vn2L0wemE1PgICLr{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Vn2L0wemE1PgICLr .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Vn2L0wemE1PgICLr .error-icon{fill:#552222;}#mermaid-svg-Vn2L0wemE1PgICLr .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Vn2L0wemE1PgICLr .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Vn2L0wemE1PgICLr .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Vn2L0wemE1PgICLr .marker.cross{stroke:#333333;}#mermaid-svg-Vn2L0wemE1PgICLr svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Vn2L0wemE1PgICLr p{margin:0;}#mermaid-svg-Vn2L0wemE1PgICLr .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Vn2L0wemE1PgICLr .cluster-label text{fill:#333;}#mermaid-svg-Vn2L0wemE1PgICLr .cluster-label span{color:#333;}#mermaid-svg-Vn2L0wemE1PgICLr .cluster-label span p{background-color:transparent;}#mermaid-svg-Vn2L0wemE1PgICLr .label text,#mermaid-svg-Vn2L0wemE1PgICLr span{fill:#333;color:#333;}#mermaid-svg-Vn2L0wemE1PgICLr .node rect,#mermaid-svg-Vn2L0wemE1PgICLr .node circle,#mermaid-svg-Vn2L0wemE1PgICLr .node ellipse,#mermaid-svg-Vn2L0wemE1PgICLr .node polygon,#mermaid-svg-Vn2L0wemE1PgICLr .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Vn2L0wemE1PgICLr .rough-node .label text,#mermaid-svg-Vn2L0wemE1PgICLr .node .label text,#mermaid-svg-Vn2L0wemE1PgICLr .image-shape .label,#mermaid-svg-Vn2L0wemE1PgICLr .icon-shape .label{text-anchor:middle;}#mermaid-svg-Vn2L0wemE1PgICLr .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Vn2L0wemE1PgICLr .rough-node .label,#mermaid-svg-Vn2L0wemE1PgICLr .node .label,#mermaid-svg-Vn2L0wemE1PgICLr .image-shape .label,#mermaid-svg-Vn2L0wemE1PgICLr .icon-shape .label{text-align:center;}#mermaid-svg-Vn2L0wemE1PgICLr .node.clickable{cursor:pointer;}#mermaid-svg-Vn2L0wemE1PgICLr .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Vn2L0wemE1PgICLr .arrowheadPath{fill:#333333;}#mermaid-svg-Vn2L0wemE1PgICLr .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Vn2L0wemE1PgICLr .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Vn2L0wemE1PgICLr .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Vn2L0wemE1PgICLr .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Vn2L0wemE1PgICLr .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Vn2L0wemE1PgICLr .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Vn2L0wemE1PgICLr .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Vn2L0wemE1PgICLr .cluster text{fill:#333;}#mermaid-svg-Vn2L0wemE1PgICLr .cluster span{color:#333;}#mermaid-svg-Vn2L0wemE1PgICLr div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Vn2L0wemE1PgICLr .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Vn2L0wemE1PgICLr rect.text{fill:none;stroke-width:0;}#mermaid-svg-Vn2L0wemE1PgICLr .icon-shape,#mermaid-svg-Vn2L0wemE1PgICLr .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Vn2L0wemE1PgICLr .icon-shape p,#mermaid-svg-Vn2L0wemE1PgICLr .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Vn2L0wemE1PgICLr .icon-shape .label rect,#mermaid-svg-Vn2L0wemE1PgICLr .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Vn2L0wemE1PgICLr .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Vn2L0wemE1PgICLr .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Vn2L0wemE1PgICLr :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 调度开始
预选阶段 Filtering
优选阶段 Scoring
选定节点 Binding
资源检查
策略检查
拓扑检查
打分排序
选择最优

2.2 预选阶段(Filtering)

检查节点是否满足 Pod 的基本要求:

检查项 说明
资源检查 CPU、内存是否充足
端口检查 hostPort 是否冲突
节点选择器 nodeSelector 是否匹配
污点检查 是否有未容忍的污点
亲和性检查 nodeAffinity 是否满足

2.3 优选阶段(Scoring)

对通过预选的节点进行打分:

评分项 说明 权重
资源均衡 资源使用越均衡分越高
亲和性 满足亲和性加分
拓扑分布 满足拓扑分布加分
镜像本地性 节点有镜像加分

2.4 调度结果

bash 复制代码
# 查看调度结果
kubectl get pod <pod-name> -o wide
# NODE 列显示调度到的节点

# 查看调度事件
kubectl describe pod <pod-name> | grep -A10 Events

三、节点选择策略

3.1 nodeSelector(简单选择)

yaml 复制代码
apiVersion: v1
kind: Pod
metadata:
  name: node-selector-pod
spec:
  nodeSelector:           # 简单标签选择
    disktype: ssd
    environment: production
  containers:
  - name: app
    image: nginx

给节点打标签

bash 复制代码
# 添加标签
kubectl label nodes <node-name> disktype=ssd

# 查看节点标签
kubectl get nodes --show-labels

# 删除标签
kubectl label nodes <node-name> disktype-

3.2 nodeName(直接指定)

yaml 复制代码
apiVersion: v1
kind: Pod
metadata:
  name: node-name-pod
spec:
  nodeName: node-1        # 直接指定节点
  containers:
  - name: app
    image: nginx

注意

  • 绕过调度器
  • 节点不存在则 Pod 失败
  • 适合调试场景

3.3 nodeAffinity(高级选择)

yaml 复制代码
apiVersion: v1
kind: Pod
metadata:
  name: node-affinity-pod
spec:
  affinity:
    nodeAffinity:
      # 硬性要求(必须满足)
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: disktype
            operator: In
            values: [ssd, nvme]
          - key: zone
            operator: NotIn
            values: [zone-a]
      
      # 软性偏好(尽量满足)
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 80          # 权重 1-100
        preference:
          matchExpressions:
          - key: performance
            operator: In
            values: [high]
      - weight: 20
        preference:
          matchExpressions:
          - key: region
            operator: In
            values: [east]
  containers:
  - name: app
    image: nginx

操作符

操作符 说明
In 在列表中
NotIn 不在列表中
Exists 键存在
DoesNotExist 键不存在
Gt 大于
Lt 小于

四、亲和性与反亲和性

4.1 Pod 亲和性(Pod Affinity)

将 Pod 调度到与其他 Pod 相同的节点/区域。

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      affinity:
        podAffinity:
          # 硬性要求:必须与 cache Pod 在同一节点
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: cache
            topologyKey: kubernetes.io/hostname
          
          # 软性偏好:尽量与 db Pod 在同一区域
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  app: db
              topologyKey: topology.kubernetes.io/zone
      containers:
      - name: web
        image: nginx

4.2 Pod 反亲和性(Pod Anti-Affinity)

将 Pod 分散到不同节点/区域。

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      affinity:
        podAntiAffinity:
          # 硬性要求:必须分散到不同节点
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: web
            topologyKey: kubernetes.io/hostname
          
          # 软性偏好:尽量分散到不同区域
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  app: web
              topologyKey: topology.kubernetes.io/zone
      containers:
      - name: web
        image: nginx

4.3 topologyKey 说明

topologyKey 说明
kubernetes.io/hostname 节点级别
topology.kubernetes.io/zone 可用区级别
topology.kubernetes.io/region 区域级别

五、污点与容忍

5.1 污点(Taints)

污点用于排斥 Pod 调度到特定节点。

bash 复制代码
# 添加污点
kubectl taint nodes <node-name> key=value:effect

# 示例
kubectl taint nodes node-1 dedicated=production:NoSchedule
kubectl taint nodes node-2 gpu=true:NoExecute
kubectl taint nodes node-3 maintenance=true:PreferNoSchedule

# 删除污点
kubectl taint nodes <node-name> key=value:effect-
kubectl taint nodes node-1 dedicated:NoSchedule-

5.2 污点效果(Effect)

效果 说明
NoSchedule 不调度新 Pod
PreferNoSchedule 尽量不调度
NoExecute 不调度且驱逐现有 Pod

5.3 容忍(Tolerations)

Pod 通过容忍来"接受"节点的污点。

yaml 复制代码
apiVersion: v1
kind: Pod
metadata:
  name: toleration-pod
spec:
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "production"
    effect: "NoSchedule"
  - key: "gpu"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 3600    # 容忍 3600 秒后驱逐
  containers:
  - name: app
    image: nginx

5.4 典型应用场景

场景 配置
专用节点 污点 + 容忍
GPU 节点 污点 + 容忍
维护节点 NoExecute + tolerationSeconds
隔离节点 NoSchedule

六、拓扑分布约束

6.1 TopologySpreadConstraints

更灵活地控制 Pod 在拓扑域中的分布。

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 6
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      topologySpreadConstraints:
      # 按可用区分布
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app: web
      
      # 按节点分布
      - maxSkew: 1
        topologyKey: kubernetes.io/hostname
        whenUnsatisfiable: ScheduleAnyway
        labelSelector:
          matchLabels:
            app: web
      
      containers:
      - name: web
        image: nginx

6.2 关键参数

参数 说明
maxSkew 最大不均衡度
topologyKey 拓扑域键
whenUnsatisfiable 不满足时的策略
labelSelector Pod 选择器

6.3 whenUnsatisfiable 策略

策略 说明
DoNotSchedule 不调度(硬性)
ScheduleAnyway 仍然调度(软性)

七、实战:生产级调度配置

7.1 完整示例:高可用 Web 应用

yaml 复制代码
apiVersion: apps/v1
kind: Deployment
metadata:
  name: production-web
  namespace: production
spec:
  replicas: 6
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      # 节点亲和性:优先 SSD 节点
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: disktype
                operator: In
                values: [ssd]
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 80
            preference:
              matchExpressions:
              - key: performance
                operator: In
                values: [high]
        
        # Pod 反亲和性:分散到不同节点
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: web
            topologyKey: kubernetes.io/hostname
      
      # 拓扑分布:均匀分布到可用区
      topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app: web
      
      # 容忍:允许调度到有特定污点的节点
      tolerations:
      - key: "production"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
      
      containers:
      - name: web
        image: nginx:1.25.3
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "256Mi"

7.2 GPU 工作负载调度

yaml 复制代码
apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
spec:
  # 节点选择:GPU 节点
  nodeSelector:
    gpu: "true"
  
  # 容忍 GPU 污点
  tolerations:
  - key: "gpu"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  
  containers:
  - name: cuda-app
    image: cuda-app:1.0
    resources:
      limits:
        nvidia.com/gpu: 1     # 请求 GPU 资源

7.3 有状态应用调度

yaml 复制代码
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  serviceName: mysql
  replicas: 3
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      # 反亲和性:每个节点一个实例
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: mysql
            topologyKey: kubernetes.io/hostname
      
      # 节点选择:SSD 节点
      nodeSelector:
        disktype: ssd
      
      containers:
      - name: mysql
        image: mysql:8.0
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
  
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: fast-ssd
      resources:
        requests:
          storage: 100Gi

八、常用命令速查

8.1 节点标签操作

bash 复制代码
# 查看节点标签
kubectl get nodes --show-labels

# 添加标签
kubectl label nodes <node-name> disktype=ssd

# 删除标签
kubectl label nodes <node-name> disktype-

# 查看特定节点详情
kubectl describe node <node-name>

8.2 污点操作

bash 复制代码
# 查看污点
kubectl describe node <node-name> | grep Taints

# 添加污点
kubectl taint nodes <node-name> key=value:NoSchedule

# 删除污点
kubectl taint nodes <node-name> key:NoSchedule-

# 删除所有污点
kubectl taint nodes <node-name> --all-

8.3 调度调试

bash 复制代码
# 查看 Pod 调度结果
kubectl get pod <pod-name> -o wide

# 查看调度事件
kubectl describe pod <pod-name> | grep -A10 Events

# 查看未调度的 Pod
kubectl get pods --field-selector=status.phase=Pending

# 查看节点资源使用
kubectl top nodes

# 查看调度器日志
kubectl logs -n kube-system kube-scheduler-<node-name>

8.4 模拟调度

bash 复制代码
# 使用 --dry-run 测试
kubectl apply -f pod.yaml --dry-run=client

# 查看调度配置
kubectl get configmap kube-scheduler -n kube-system -o yaml

九、最佳实践与避坑指南

9.1 必须遵守的原则

原则 说明 示例
使用反亲和性 提高可用性 分散到不同节点
配置拓扑分布 跨可用区分布 topologySpreadConstraints
合理使用污点 专用节点隔离 GPU 节点
设置资源请求 确保调度成功 requests
避免 nodeName 绕过调度器 调试除外

9.2 常见问题与解决

问题 原因 解决方案
Pod 一直 Pending 资源不足 增加节点或减少请求
调度不均衡 缺少拓扑约束 添加 topologySpreadConstraints
Pod 被驱逐 污点不匹配 添加容忍
亲和性冲突 条件矛盾 检查 affinity 配置
调度器性能 大量 Pod 调度 优化调度策略

9.3 调度策略优化

yaml 复制代码
# 1. 使用软性约束(避免调度失败)
affinity:
  nodeAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      preference:
        matchExpressions:
        - key: performance
          operator: In
          values: [high]

# 2. 合理设置资源请求(避免过度分配)
resources:
  requests:
    cpu: "100m"    # 不要设置过高
    memory: "128Mi"

# 3. 使用 PodDisruptionBudget(保证可用性)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web

9.4 多租户调度

yaml 复制代码
# 为不同团队配置不同的调度策略
# 团队 A:高性能节点
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-a-quota
  namespace: team-a
spec:
  hard:
    requests.cpu: "100"
    requests.memory: "200Gi"

# 使用节点池隔离
kubectl label nodes node-1 node-pool=team-a
kubectl label nodes node-2 node-pool=team-b

十、总结

10.1 核心要点回顾

  1. 调度器两阶段:预选过滤 + 优选打分
  2. 节点选择:nodeSelector 简单,nodeAffinity 灵活
  3. Pod 亲和性:控制 Pod 间的关系
  4. 污点容忍:节点专用化
  5. 拓扑分布:跨故障域分布

10.2 调度策略选择建议

场景 推荐策略
高可用 反亲和性 + 拓扑分布
性能优化 节点亲和性 + 专用节点
成本控制 软性约束 + 合理请求
多租户 节点池 + 污点隔离

10.3 学习路线

复制代码
调度器基础 → 亲和性 → 污点容忍 → 拓扑分布
    ↓
自定义调度器 → 调度框架 → 性能优化

10.4 下一步学习

  1. 自定义调度器:实现特定调度逻辑
  2. 调度框架:扩展调度器插件
  3. Descheduler:重新平衡 Pod
  4. Cluster Autoscaler:自动扩缩节点
  5. Karpenter:新一代节点自动伸缩

附录:完整示例汇总

生产级调度完整配置

yaml 复制代码
# 1. 高可用 Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ha-web-app
  namespace: production
spec:
  replicas: 6
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      # 节点亲和性
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values: [linux]
        
        # Pod 反亲和性
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: web
            topologyKey: kubernetes.io/hostname
      
      # 拓扑分布
      topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app: web
      
      # 容忍
      tolerations:
      - key: "production"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
      
      containers:
      - name: web
        image: nginx:1.25.3
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "256Mi"

验证测试

bash 复制代码
# 1. 创建 Deployment
kubectl apply -f scheduling-demo.yaml

# 2. 查看 Pod 分布
kubectl get pods -n production -o wide

# 3. 验证反亲和性(每个节点一个 Pod)
kubectl get pods -n production -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.nodeName}{"\n"}{end}'

# 4. 验证拓扑分布
kubectl get nodes -L topology.kubernetes.io/zone

# 5. 测试调度失败场景
# 创建超过节点数的副本
kubectl scale deployment ha-web-app --replicas=20
# 观察 Pending Pod
kubectl get pods -n production --field-selector=status.phase=Pending

相关推荐
IT大白鼠5 小时前
Ingress 与 Ingress-Controller:K8s 七层 HTTP 反向代理技术详解与实践
http·容器·kubernetes
java_logo6 小时前
Docker 部署 Rocky Linux:轻松搭建 RHEL 兼容企业级基础镜像平台
linux·docker·容器·rocky linux·基础镜像·轩辕镜像·rhel 兼容
阿里云云原生9 小时前
论坛剧透丨当企业有 1000 个智能体后怎么跑怎么管?
云原生
Henry-SAP10 小时前
GPT-6Astra开启AI自主执行新时代
人工智能·云原生·sap·erp
IT大白鼠12 小时前
Istio 结合 K8s 实现流量管理:服务网格灰度、熔断、限流实战
云原生·kubernetes·istio
鹤落晴春12 小时前
【K8s】认识Kubernetes API:从CRD出发的一次梳理
云原生·容器·kubernetes
yanwumuxi14 小时前
Docker记录:误删宿主机挂载目录导致 PostgreSQL 数据丢失
docker·postgresql·容器
cui_hao_nan16 小时前
Docker 学习 5
docker·容器
A黄俊辉A16 小时前
两分钟上手docker
运维·docker·容器
努力努力再努力wz18 小时前
【Docker入门系列】:从架构演进到容器化:一文建立 Docker、虚拟化与 Namespace 的底层心智模型
运维·开发语言·数据结构·c++·docker·容器·架构