内核版本:linux-5.15.140
涉及文件:kernel/sched/fair.c
前言
上一篇(第 38 篇)我们把 update_sd_lb_stats() 的两级账本、group_type 六级分类和 update_sd_pick_busiest() 四层竞选拆完了。到这一步,sds.busiest_stat 里躺着最忙组的完整画像,但"画像"还不是"判决"------本地组该不该从这个最忙组拉任务、拉多少、从组里哪个 CPU 拉,是三个独立的问题。
本篇把这三个问题一次拆完,调用关系如下:
load_balance() kernel/sched/fair.c:10163
├── find_busiest_group(&env) kernel/sched/fair.c:9805 该不该搬:决策矩阵
│ └── calculate_imbalance() kernel/sched/fair.c:9622 搬多少:四种搬运模式
└── find_busiest_queue(&env, group) kernel/sched/fair.c:9934 从哪搬:四种选队列标尺
本篇是负载均衡专题的收官前哨:走出 find_busiest_queue() 之后,load_balance() 拿着 env.src_rq 进入 detach_tasks()/attach_tasks() 两段式搬运(见第 37 篇主流程)。
一、决策矩阵:内核注释里的"真值表"
在看代码前,先看内核作者写在 find_busiest_group() 上方的注释,这张表就是整个函数的说明书(kernel/sched/fair.c:9774):
c
/*
* Decision matrix according to the local and busiest group type:
*
* busiest \ local has_spare fully_busy misfit asym imbalanced overloaded
* has_spare nr_idle balanced N/A N/A balanced balanced
* fully_busy nr_idle nr_idle N/A N/A balanced balanced
* misfit_task force N/A N/A N/A force force
* asym_packing force force N/A N/A force force
* imbalanced force force N/A N/A force force
* overloaded force force N/A N/A force avg_load
*
* N/A : Not Applicable because already filtered while updating
* statistics.
* balanced : The system is balanced for these 2 groups.
* force : Calculate the imbalance as load migration is probably needed.
* avg_load : Only if imbalance is significant enough.
* nr_idle : dst_cpu is not busy and the number of idle CPUs is quite
* different in groups.
*/
读法说明:
- N/A :
misfit/asym列不存在------本地组是 misfit 或 asym 时,本地组自己就会成为被拉的"最忙组",update_sd_pick_busiest()阶段早就把这种组合过滤了(见第 38 篇四层竞选的一票否决)。 - balanced:直接判"没失衡",返回 NULL。
- force :绕过精细比较,强制进入
calculate_imbalance()计算搬运量。 - avg_load :两边都过载时才动用平均负载精细比较,且要超过
imbalance_pct滞回线才动手。 - nr_idle:没过载的场景下,按"空闲 CPU 数差异"来均衡。
这张表在 5.15 内核里就是负载均衡的顶层策略,后面的代码都是它的展开。
二、find_busiest_group():决策矩阵的代码化
完整源码(kernel/sched/fair.c:9805):
c
static struct sched_group *find_busiest_group(struct lb_env *env)
{
struct sg_lb_stats *local, *busiest;
struct sd_lb_stats sds;
init_sd_lb_stats(&sds);
/*
* Compute the various statistics relevant for load balancing at
* this level.
*/
update_sd_lb_stats(env, &sds);
if (sched_energy_enabled()) {
struct root_domain *rd = env->dst_rq->rd;
if (rcu_dereference(rd->pd) && !READ_ONCE(rd->overutilized))
goto out_balanced;
}
local = &sds.local_stat;
busiest = &sds.busiest_stat;
/* There is no busy sibling group to pull tasks from */
if (!sds.busiest)
goto out_balanced;
/* Misfit tasks should be dealt with regardless of the avg load */
if (busiest->group_type == group_misfit_task)
goto force_balance;
/* ASYM feature bypasses nice load balance check */
if (busiest->group_type == group_asym_packing)
goto force_balance;
/*
* If the busiest group is imbalanced the below checks don't
* work because they assume all things are equal, which typically
* isn't true due to cpus_ptr constraints and the like.
*/
if (busiest->group_type == group_imbalanced)
goto force_balance;
/*
* If the local group is busier than the selected busiest group
* don't try and pull any tasks.
*/
if (local->group_type > busiest->group_type)
goto out_balanced;
/*
* When groups are overloaded, use the avg_load to ensure fairness
* between tasks.
*/
if (local->group_type == group_overloaded) {
/*
* If the local group is more loaded than the selected
* busiest group don't try and pull any tasks.
*/
if (local->avg_load >= busiest->avg_load)
goto out_balanced;
/* XXX broken for overlapping NUMA groups */
sds.avg_load = (sds.total_load * SCHED_CAPACITY_SCALE) /
sds.total_capacity;
/*
* Don't pull any tasks if this group is already above the
* domain average load.
*/
if (local->avg_load >= sds.avg_load)
goto out_balanced;
/*
* If the busiest group is more loaded, use imbalance_pct to be
* conservative.
*/
if (100 * busiest->avg_load <=
env->sd->imbalance_pct * local->avg_load)
goto out_balanced;
}
/* Try to move all excess tasks to child's sibling domain */
if (sds.prefer_sibling && local->group_type == group_has_spare &&
busiest->sum_nr_running > local->sum_nr_running + 1)
goto force_balance;
if (busiest->group_type != group_overloaded) {
if (env->idle == CPU_NOT_IDLE)
/*
* If the busiest group is not overloaded (and as a
* result the local one too) but this CPU is already
* busy, let another idle CPU try to pull task.
*/
goto out_balanced;
if (busiest->group_weight > 1 &&
local->idle_cpus <= (busiest->idle_cpus + 1))
/*
* If the busiest group is not overloaded
* and there is no imbalance between this and busiest
* group wrt idle CPUs, it is balanced. The imbalance
* becomes significant if the diff is greater than 1
* otherwise we might end up to just move the imbalance
* on another group. Of course this applies only if
* there is more than 1 CPU per group.
*/
goto out_balanced;
if (busiest->sum_h_nr_running == 1)
/*
* busiest doesn't have any tasks waiting to run
*/
goto out_balanced;
}
force_balance:
/* Looks like there is an imbalance. Compute it */
calculate_imbalance(env, &sds);
return env->imbalance ? sds.busiest : NULL;
out_balanced:
env->imbalance = 0;
return NULL;
}
逐段拆解。
2.1 统计与 EAS 让路
c
init_sd_lb_stats(&sds);
update_sd_lb_stats(env, &sds);
这两行是第 38 篇的全部内容:初始化"最坏初值"、遍历组环累加统计、选出最忙组。
c
if (sched_energy_enabled()) {
struct root_domain *rd = env->dst_rq->rd;
if (rcu_dereference(rd->pd) && !READ_ONCE(rd->overutilized))
goto out_balanced;
}
若 EAS(Energy Aware Scheduling)已启用、系统里存在 perf domain(rd->pd 非空)且整个 root domain 尚未过载(rd->overutilized 为 0),普通负载均衡直接让路------任务放置交给能量模型去决策。overutilized 正是第 38 篇里 update_sd_lb_stats() 在根域层回写的指示器:一旦置位,EAS 失效,传统负载均衡重新接管。
2.2 三个 force_balance 入口:不看负载也要搬
c
if (!sds.busiest)
goto out_balanced;
/* Misfit tasks should be dealt with regardless of the avg load */
if (busiest->group_type == group_misfit_task)
goto force_balance;
/* ASYM feature bypasses nice load balance check */
if (busiest->group_type == group_asym_packing)
goto force_balance;
三种类型直接跳 force_balance,对应决策矩阵里的三行 force:
- group_misfit_task:最忙组里有任务在低算力 CPU 上"穿小鞋"。这种任务的 util 超过了所在 CPU 的容量,无论组平均负载多低都应该搬到更大核上,注释原文"regardless of the avg load"。
- group_asym_packing:异构打包场景,低编号(更优先)CPU 空着,高编号 CPU 上有活,需要把负载往 preferred CPU 上收。
- group_imbalanced (下一段代码):最忙组已经因亲和性约束(
cpus_ptr)等发生过"搬不动"的失败,组级平均已经失真。注释说得很直白:下面的检查都假设"各组条件相等",而 imbalanced 组恰恰不满足这个前提,所以只能"try to move any task",先搬一个是一个,下次均衡再慢慢收敛。
2.3 本地更忙与过载精细比较
c
if (local->group_type > busiest->group_type)
goto out_balanced;
group_type 按拉取优先级排序(见第 38 篇),本地类型数值更大说明本地更"忙",没资格拉人------这对应决策矩阵对角线以下的 balanced 格子。
c
if (local->group_type == group_overloaded) {
if (local->avg_load >= busiest->avg_load)
goto out_balanced;
/* XXX broken for overlapping NUMA groups */
sds.avg_load = (sds.total_load * SCHED_CAPACITY_SCALE) /
sds.total_capacity;
if (local->avg_load >= sds.avg_load)
goto out_balanced;
if (100 * busiest->avg_load <=
env->sd->imbalance_pct * local->avg_load)
goto out_balanced;
}
两边都过载时(overloaded × overloaded,矩阵右下角),动用三道闸门:
- 组间比较:本地平均负载不低于最忙组,不搬。
- 域平均闸门 :
sds.avg_load是整个调度域的加权平均负载(total_load × 1024 / total_capacity)。注释里的XXX broken for overlapping NUMA groups承认这个算法在 NUMA 组重叠时是错的------重叠组的负载会被重复计入total_load。本地已经高于域平均还去拉任务,只会拉高自己、加剧全局不均。 - imbalance_pct 滞回 :
100 × busiest_avg ≤ imbalance_pct × local_avg(默认 117,见第 36 篇sd_init)不搬。也就是说最忙组要比本地忙出 17% 以上才值得动手------防止两个负载相近的组来回倒腾任务。
2.4 prefer_sibling:往子域兄弟组里赶人
c
/* Try to move all excess tasks to child's sibling domain */
if (sds.prefer_sibling && local->group_type == group_has_spare &&
busiest->sum_nr_running > local->sum_nr_running + 1)
goto force_balance;
prefer_sibling 是 MC 层对 DIE 层的暗示:LLC 内还有富余,请把 DIE 级别的多余任务往我这里搬。条件是本地有富余容量、且最忙组任务数比本地多出不止 1 个(+1 的余量避免为单个任务跨 LLC 折腾)。
2.5 未过载时的三道否决
c
if (busiest->group_type != group_overloaded) {
if (env->idle == CPU_NOT_IDLE)
goto out_balanced;
if (busiest->group_weight > 1 &&
local->idle_cpus <= (busiest->idle_cpus + 1))
goto out_balanced;
if (busiest->sum_h_nr_running == 1)
goto out_balanced;
}
最忙组顶多是 fully_busy(含 has_spare,因为本地更忙的组合已在前面的类型比较里被否决)时:
- 本 CPU 不空闲就别掺和 :
CPU_NOT_IDLE说明发起均衡的 CPU 自己忙着,这种"未过载"级别的小失衡留给空闲 CPU 去处理。 - 空闲 CPU 数差异不显著 :组内多于 1 个 CPU 时,本地空闲数 ≤ 最忙组空闲数 +1 视为均衡。注释解释了
+1:差异为 1 就搬,只会把失衡从一个组转移到另一个组。这对应矩阵左上角的nr_idle格子。 - 最忙组只有一个任务 :
sum_h_nr_running == 1说明组里那一个任务正在某颗 CPU 上跑得好好的,没有排队的任务可拉------硬拉就得靠主动均衡抢(那是need_active_balance()的领域,见第 37 篇)。
2.6 收尾
c
force_balance:
/* Looks like there is an imbalance. Compute it */
calculate_imbalance(env, &sds);
return env->imbalance ? sds.busiest : NULL;
out_balanced:
env->imbalance = 0;
return NULL;
走到 force_balance 只说明"值得算一算",最终 calculate_imbalance() 算出的 env->imbalance 若为 0(比如 NUMA 小失衡被容忍,见下文),照样返回 NULL。
三、calculate_imbalance():把"该不该"翻译成"搬多少"
find_busiest_group() 只回答了"要不要",calculate_imbalance()(kernel/sched/fair.c:9622)负责设定两个输出:env->migration_type(用什么标尺衡量搬运量)和 env->imbalance(搬运量目标值)。
migration_type 的四种取值(kernel/sched/fair.c:7862,第 37 篇已随 lb_env 拆过,这里复习含义):
c
enum migration_type {
migrate_load = 0,
migrate_util,
migrate_task,
migrate_misfit
};
3.1 三个特判:imbalance 直接给死
c
if (busiest->group_type == group_misfit_task) {
/* Set imbalance to allow misfit tasks to be balanced. */
env->migration_type = migrate_misfit;
env->imbalance = 1;
return;
}
if (busiest->group_type == group_asym_packing) {
/*
* In case of asym capacity, we will try to migrate all load to
* the preferred CPU.
*/
env->migration_type = migrate_task;
env->imbalance = busiest->sum_h_nr_running;
return;
}
if (busiest->group_type == group_imbalanced) {
/*
* In the group_imb case we cannot rely on group-wide averages
* to ensure CPU-load equilibrium, try to move any task to fix
* the imbalance. The next load balance will take care of
* balancing back the system.
*/
env->migration_type = migrate_task;
env->imbalance = 1;
return;
}
- misfit:目标就是"把那个大任务搬走",一次搬一个,量纲是任务数。
- asym_packing :
imbalance直接给最忙组的全部任务数------把负载整个收拢到 preferred CPU 上,宁可多搬。 - imbalanced:组级平均已失真,只求搬动任意一个任务破局,"下次负载均衡再负责把系统搬回去"。
3.2 本地有富余(group_has_spare):按容量或按任务数填坑
c
if (local->group_type == group_has_spare) {
if ((busiest->group_type > group_fully_busy) &&
!(env->sd->flags & SD_SHARE_PKG_RESOURCES)) {
/*
* If busiest is overloaded, try to fill spare
* capacity. This might end up creating spare
* capacity in busiest or busiest still being
* overloaded but there is no simple way to directly
* compute the amount of load to migrate in order
* to balance the system.
*/
env->migration_type = migrate_util;
env->imbalance = max(local->group_capacity, local->group_util) -
local->group_util;
/*
* In some cases, the group's utilization is max or even
* higher than capacity because of migrations but the
* local CPU is (newly) idle. There is at least one
* waiting task in this overloaded busiest group. Let's
* try to pull it.
*/
if (env->idle != CPU_NOT_IDLE && env->imbalance == 0) {
env->migration_type = migrate_task;
env->imbalance = 1;
}
return;
}
最忙组过载、本地有富余、且当前域不共享 LLC(SD_SHARE_PKG_RESOURCES 未设,比如 DIE 层)时:目标是填满本地的富余容量 ,imbalance = max(group_capacity, group_util) - group_util。取 max 是防御性写法------迁移中的瞬时尖峰可能让 group_util 超过容量,直接相减会下溢出无符号数。量纲是 util(PELT util 信号,见第 34 篇)。注释也承认这只是近似:没法一步算出精确的搬运量,可能搬完最忙组还是有富余或还过载,靠多轮均衡收敛。
兜底:本地组 util 已顶格但本 CPU 刚刚空闲(newly idle 场景),而最忙组还有排队任务------那就按任务数搬一个。
c
if (busiest->group_weight == 1 || sds->prefer_sibling) {
unsigned int nr_diff = busiest->sum_nr_running;
/*
* When prefer sibling, evenly spread running tasks on
* groups.
*/
env->migration_type = migrate_task;
lsub_positive(&nr_diff, local->sum_nr_running);
env->imbalance = nr_diff >> 1;
} else {
/*
* If there is no overload, we just want to even the number of
* idle cpus.
*/
env->migration_type = migrate_task;
env->imbalance = max_t(long, 0, (local->idle_cpus -
busiest->idle_cpus) >> 1);
}
两边都没过载时的两种均分策略,量纲都是任务数:
- 单 CPU 组或 prefer_sibling :按任务数差对半分------
nr_diff = busiest 任务数 - local 任务数(lsub_positive是"饱和减法",见第 35 篇,减不够就归零),再右移一位。prefer_sibling 场景下这就是"把 LLC 内的任务摊匀"。 - 一般情况 :按空闲 CPU 数差对半分------
local 空闲数 - busiest 空闲数的差取半。没过载时负载不是问题,空闲 CPU 数才是"还有多少坑"的直接度量,对应矩阵的nr_idle格子。
3.3 NUMA 小失衡容忍
c
/* Consider allowing a small imbalance between NUMA groups */
if (env->sd->flags & SD_NUMA) {
env->imbalance = adjust_numa_imbalance(env->imbalance,
local->sum_nr_running + 1, local->group_weight);
}
return;
}
跨 NUMA 节点搬任务要付出访存延迟的代价,所以 NUMA 域允许"适当穷忍"(kernel/sched/fair.c:9598):
c
#define NUMA_IMBALANCE_MIN 2
static inline long adjust_numa_imbalance(int imbalance,
int dst_running, int dst_weight)
{
if (!allow_numa_imbalance(dst_running, dst_weight))
return imbalance;
/*
* Allow a small imbalance based on a simple pair of communicating
* tasks that remain local when the destination is lightly loaded.
*/
if (imbalance <= NUMA_IMBALANCE_MIN)
return 0;
return imbalance;
}
allow_numa_imbalance()(kernel/sched/fair.c:9292):
c
/*
* Allow a NUMA imbalance if busy CPUs is less than 25% of the domain.
* This is an approximation as the number of running tasks may not be
* related to the number of busy CPUs due to sched_setaffinity.
*/
static inline bool
allow_numa_imbalance(unsigned int running, unsigned int weight)
{
return (running < (weight >> 2));
}
规则:目的地轻载(任务数 < 组 CPU 数的 1/4)且算出的失衡量 ≤ 2 时,直接归零不搬。注释点出动机------一对互相通信的任务(典型如生产者/消费者)为一个任务的"账面均衡"被拆到两个节点,得不偿失;目的地反正闲着,让它稍微少干点没关系。
3.4 本地满载还硬接:overloaded 的精细公式
c
/*
* Local is fully busy but has to take more load to relieve the
* busiest group
*/
if (local->group_type < group_overloaded) {
/*
* Local will become overloaded so the avg_load metrics are
* finally needed.
*/
local->avg_load = (local->group_load * SCHED_CAPACITY_SCALE) /
local->group_capacity;
/*
* If the local group is more loaded than the selected
* busiest group don't try to pull any tasks.
*/
if (local->avg_load >= busiest->avg_load) {
env->imbalance = 0;
return;
}
sds->avg_load = (sds->total_load * SCHED_CAPACITY_SCALE) /
sds->total_capacity;
/*
* If the local group is more loaded than the average system
* load, don't try to pull any tasks.
*/
if (local->avg_load >= sds->avg_load) {
env->imbalance = 0;
return;
}
}
本地 fully_busy(或更轻,但前面的路径已处理)去接济过载的最忙组时,本地将会变成 overloaded------此时"avg_load 只有在过载时才计算"的约定(见第 38 篇update_sg_lb_stats)不够用了,先补算本地的平均负载,再走与 find_busiest_group() 2.3 节同款的两道闸门(不低于最忙组、不低于域平均),不过关就作罢。
c
/*
* Both group are or will become overloaded and we're trying to get all
* the CPUs to the average_load, so we don't want to push ourselves
* above the average load, nor do we wish to reduce the max loaded CPU
* below the average load. At the same time, we also don't want to
* reduce the group load below the group capacity. Thus we look for
* the minimum possible imbalance.
*/
env->migration_type = migrate_load;
env->imbalance = min(
(busiest->avg_load - sds->avg_load) * busiest->group_capacity,
(sds->avg_load - local->avg_load) * local->group_capacity
) / SCHED_CAPACITY_SCALE;
终局公式,量纲是 load:把两组都拉向域平均,搬运量取两个方向约束的较小值:
(busiest_avg - 域平均) × busiest 容量:最忙组高于域平均的"超额量",搬多了会把最忙组拉到域平均之下;(域平均 - local_avg) × local 容量:本地低于域平均的"欠额量",搬多了会把本地顶到域平均之上。
两边都不能越界,所以取 min。除以 SCHED_CAPACITY_SCALE(1024)把 avg_load 的定标换算回 load 的定标------avg_load = group_load × 1024 / group_capacity,反解回来正好约掉缩放因子。
四、find_busiest_queue():组内选队列,一把钥匙开一把锁
组定了、量定了,最后在最忙组的 CPU 里挑出具体的 env.src_rq(kernel/sched/fair.c:9934):
c
/*
* find_busiest_queue - find the busiest runqueue among the CPUs in the group.
*/
static struct rq *find_busiest_queue(struct lb_env *env,
struct sched_group *group)
{
struct rq *busiest = NULL, *rq;
unsigned long busiest_util = 0, busiest_load = 0, busiest_capacity = 1;
unsigned int busiest_nr = 0;
int i;
for_each_cpu_and(i, sched_group_span(group), env->cpus) {
unsigned long capacity, load, util;
unsigned int nr_running;
enum fbq_type rt;
rq = cpu_rq(i);
rt = fbq_classify_rq(rq);
/*
* We classify groups/runqueues into three groups:
* - regular: there are !numa tasks
* - remote: there are numa tasks that run on the 'wrong' node
* - all: there is no distinction
*
* In order to avoid migrating ideally placed numa tasks,
* ignore those when there's better options.
*
* If we ignore the actual busiest queue to migrate another
* task, the next balance pass can still reduce the busiest
* queue by moving tasks around inside the node.
*
* If we cannot move enough load due to this classification
* the next pass will adjust the group classification and
* allow migration of more tasks.
*
* Both cases only affect the total convergence complexity.
*/
if (rt > env->fbq_type)
continue;
nr_running = rq->cfs.h_nr_running;
if (!nr_running)
continue;
capacity = capacity_of(i);
/*
* For ASYM_CPUCAPACITY domains, don't pick a CPU that could
* eventually lead to active_balancing high->low capacity.
* Higher per-CPU capacity is considered better than balancing
* average load.
*/
if (env->sd->flags & SD_ASYM_CPUCAPACITY &&
!capacity_greater(capacity_of(env->dst_cpu), capacity) &&
nr_running == 1)
continue;
switch (env->migration_type) {
...
}
}
return busiest;
}
4.1 fbq 分类:别打理想放置的 NUMA 任务的主意
队列级分类 fbq_classify_rq()(kernel/sched/fair.c:9105):
c
enum fbq_type { regular, remote, all }; /* kernel/sched/fair.c:7822 */
static inline enum fbq_type fbq_classify_rq(struct rq *rq)
{
if (rq->nr_running > rq->nr_numa_running)
return regular;
if (rq->nr_running > rq->nr_preferred_running)
return remote;
return all;
}
- regular :队列上有非 NUMA 任务(
nr_running > nr_numa_running),随便搬,最优先。 - remote:任务都是 NUMA 任务且部分跑在"错误"节点上,搬它们也无所谓。
- all:任务全是 NUMA 任务且都放在首选节点上------理想放置,能不碰就不碰。
过滤规则 rt > env->fbq_type 配合 env->fbq_type 的设定:lb_env 初始化为 all(kernel/sched/fair.c:10184),随后在 update_sd_lb_stats() 里被最忙组的组级分类覆盖(kernel/sched/fair.c:9577):
c
env->fbq_type = fbq_classify_group(&sds->busiest_stat);
组级分类(fbq_classify_group,kernel/sched/fair.c:9096)与队列级同构。效果是:最忙组里存在 regular 队列时,只考虑 regular 队列;连 remote 都没有才轮到 all。注释说明:为此放过真正的最忙队列没关系------下一轮均衡会在节点内部继续消化,最坏只影响收敛速度,不影响正确性。注意这套分类只在 CONFIG_NUMA_BALANCING 开启时有效,否则一律返回 regular(kernel/sched/fair.c:9119)。
4.2 异构域的单任务保护
c
if (env->sd->flags & SD_ASYM_CPUCAPACITY &&
!capacity_greater(capacity_of(env->dst_cpu), capacity) &&
nr_running == 1)
continue;
异构算力域里,若候选 CPU 的容量不低于目的 CPU(capacity_greater 带 5% 容差,见第 38 篇)且它只跑一个任务,跳过------把这个任务从大核搬到不比它大的核上,将来还得靠主动均衡搬回来(need_active_balance() 的高→低容量场景,见第 37 篇),不如现在就不选它。注释原文:"Higher per-CPU capacity is considered better than balancing average load."
4.3 四种标尺:一把钥匙开一把锁
switch (env->migration_type) 的四个分支,每个分支的"最忙"定义都不同:
migrate_load ------ 比的是 load/capacity 比
c
case migrate_load:
/*
* When comparing with load imbalance, use cpu_load()
* which is not scaled with the CPU capacity.
*/
load = cpu_load(rq);
if (nr_running == 1 && load > env->imbalance &&
!check_cpu_capacity(rq, env->sd))
break;
/*
* For the load comparisons with the other CPUs,
* consider the cpu_load() scaled with the CPU
* capacity, so that the load can be moved away
* from the CPU that is potentially running at a
* lower capacity.
*
* Thus we're looking for max(load_i / capacity_i),
* crosswise multiplication to rid ourselves of the
* division works out to:
* load_i * capacity_j > load_j * capacity_i;
* where j is our previous maximum.
*/
if (load * busiest_capacity > busiest_load * capacity) {
busiest_load = load;
busiest_capacity = capacity;
busiest = rq;
}
break;
cpu_load()(kernel/sched/fair.c:5989)返回未按容量缩放的 cfs_rq load_avg。两个细节:
- 单任务豁免 :候选 CPU 只跑一个任务、其负载超过本次搬运量(搬不动它)、且 CPU 自身没被频率压力拖累(
check_cpu_capacity()为假,即rq->cpu_capacity × imbalance_pct ≥ cpu_capacity_orig × 100,kernel/sched/fair.c:8766)时,break跳出 switch 但不更新busiest------这个队列排不上号。反过来,若 CPU 被压得很惨(频率上不去),即使单任务也值得考虑,好歹给它换个环境。 - 交叉相乘比比值 :真正的选队列标准是
max(load_i / capacity_i)------负载要往"单位容量负载最高"的 CPU 上找。为避免除法(以及整除截断),用load_i × busiest_capacity > busiest_load × capacity_i交叉相乘比较,数学上等价。
migrate_util ------ 比的是绝对 util
c
case migrate_util:
util = cpu_util(cpu_of(rq));
/*
* Don't try to pull utilization from a CPU with one
* running task. Whatever its utilization, we will fail
* detach the task.
*/
if (nr_running <= 1)
continue;
if (busiest_util < util) {
busiest_util = util;
busiest = rq;
}
break;
填富余容量模式(3.2 节)下直接比 cpu_util() 绝对值。nr_running <= 1 的 CPU 直接跳过:唯一那个任务正在运行,detach_tasks() 搬不走正在 CPU 上跑的任务(需要走主动均衡),挑它纯属浪费。
migrate_task ------ 比的是任务数
c
case migrate_task:
if (busiest_nr < nr_running) {
busiest_nr = nr_running;
busiest = rq;
}
break;
最朴素:谁的 h_nr_running 多谁最忙。均分任务数/空闲 CPU 的场景(3.2 节后半)配这把尺。
migrate_misfit ------ 比的是 misfit 任务的负载
c
case migrate_misfit:
/*
* For ASYM_CPUCAPACITY domains with misfit tasks we
* simply seek the "biggest" misfit task.
*/
if (rq->misfit_task_load > busiest_load) {
busiest_load = rq->misfit_task_load;
busiest = rq;
}
break;
rq->misfit_task_load(kernel/sched/sched.h:1013)记录该队列上最大 misfit 任务的负载------专挑"穿小鞋"最严重的那颗 CPU,把最大的任务搬去大核。
4.4 回到 load_balance()
load_balance() 里的两连跳(kernel/sched/fair.c:10198):
c
group = find_busiest_group(&env);
if (!group) {
schedstat_inc(sd->lb_nobusyg[idle]);
goto out_balanced;
}
busiest = find_busiest_queue(&env, group);
if (!busiest) {
schedstat_inc(sd->lb_nobusyq[idle]);
goto out_balanced;
}
BUG_ON(busiest == env.dst_rq);
注意两级都可能落空:组级说有失衡(env->imbalance 非零),但组内所有队列都被 fbq 过滤或单任务保护跳过,find_busiest_queue() 返回 NULL,本轮作罢。统计上分别记入 lb_nobusyg(no busiest group)和 lb_nobusyq(no busiest queue),属于 CONFIG_SCHEDSTATS 统计桩,/proc/schedstat 里看到的就是它们(见第 37 篇)。
五、小结
- 决策矩阵是总纲 :
find_busiest_group()上方的注释表格(fair.c:9774)穷举了本地×最忙共 6×4 种有效组合的处置:force直接算、avg_load三道闸门精细比较、nr_idle按空闲 CPU 数、balanced收工、N/A已在上游过滤。 - 三类 force 场景绕过一切比较:misfit 任务(大任务穿小鞋)、asym_packing(往 preferred CPU 收拢)、imbalanced(组级统计已失真,先搬一个破局)。
- 过载×过载的三道闸门:不比最忙组忙 → 不高于域平均 → 超出 imbalance_pct(117%)滞回线,缺一不可。
- calculate_imbalance 四种模式对应四种量纲:misfit/_task 数任务、migrate_util 填富余容量、migrate_load 用 min(超额, 欠额) 公式把两组拉向域平均;NUMA 域另有小失衡容忍(目的地任务数 < CPU 数 1/4 且失衡 ≤ 2 时归零)。
- find_busiest_queue 一把钥匙开一把锁 :migrate_load 比
load/capacity交叉相乘、migrate_util 比绝对 util、migrate_task 比任务数、migrate_misfit 比 misfit 任务负载;fbq 三级分类保护理想放置的 NUMA 任务,异构域跳过"搬了也白搬"的单任务大核。
至此,load_balance() 的"选目标"阶段(组 → 队列)全部闭环。下篇进入搬运执行的最后细节:detach_tasks() 逐任务判定 can_migrate_task()------cache 热度(task_hot 与 migration_cost)、亲和性、attach_tasks() 落位,以及 sched_migrate_task 的完整一生。
系列回链 :load_balance 主流程与 lb_env/migration_type 见第 37 篇;sg_lb_stats/sd_lb_stats 账本、group_type 分类与 update_sd_pick_busiest 见第 38 篇;capacity_greater 5% 容差见第 38 篇;sched_domain/imbalance_pct 默认值见第 36 篇;PELT 三路信号见第 34 篇;lsub_positive 饱和减法与 h_nr_running 见第 35 篇。