WorkQueue 深度分析(上篇)--- 模块定位、结构、基础队列与延迟队列
基于
client-go v0.36.1util/workqueue/源码的超深度、逐行、专业级分析源文件:queue.go (370行) + delaying_queue.go (369行) + rate_limiting_queue.go (147行) + default_rate_limiters.go (295行) + metrics.go (255行) + parallelizer.go (101行)
一、模块定位
1.1 业务职责与功能定位
WorkQueue 是 Kubernetes Controller 标准编程模式中的消费侧核心组件。当 Informer 通过 ResourceEventHandler 感知到资源变更时,不是直接调谐,而是将对象的 key 入队 WorkQueue,由 Worker 协程从队列取出 key 进行调谐。
核心职责:
| 职责 | 描述 |
|---|---|
| 去重 | 同一 key 在处理期间再次 Add 不会重复入队,但标记为 dirty 确保处理完后重新处理 |
| 延迟入队 | 处理失败后延迟重试(指数退避),避免热循环 |
| 限流 | 全局令牌桶 + 单 item 指数退避双重限流 |
| 优雅关闭 | ShutDown 立即停止 / ShutDownWithDrain 等待所有 item 处理完成 |
| 指标 | 队列深度、入队延迟、处理耗时、重试次数 |
WorkQueue 解决的核心问题:Controller 调谐失败后如何安全重试?------指数退避 + 去重 + 限流。
1.2 在系统中的位置
#mermaid-svg-Rm14GQVUOysu5f5R{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Rm14GQVUOysu5f5R .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Rm14GQVUOysu5f5R .error-icon{fill:#552222;}#mermaid-svg-Rm14GQVUOysu5f5R .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Rm14GQVUOysu5f5R .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Rm14GQVUOysu5f5R .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Rm14GQVUOysu5f5R .marker.cross{stroke:#333333;}#mermaid-svg-Rm14GQVUOysu5f5R svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Rm14GQVUOysu5f5R p{margin:0;}#mermaid-svg-Rm14GQVUOysu5f5R .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Rm14GQVUOysu5f5R .cluster-label text{fill:#333;}#mermaid-svg-Rm14GQVUOysu5f5R .cluster-label span{color:#333;}#mermaid-svg-Rm14GQVUOysu5f5R .cluster-label span p{background-color:transparent;}#mermaid-svg-Rm14GQVUOysu5f5R .label text,#mermaid-svg-Rm14GQVUOysu5f5R span{fill:#333;color:#333;}#mermaid-svg-Rm14GQVUOysu5f5R .node rect,#mermaid-svg-Rm14GQVUOysu5f5R .node circle,#mermaid-svg-Rm14GQVUOysu5f5R .node ellipse,#mermaid-svg-Rm14GQVUOysu5f5R .node polygon,#mermaid-svg-Rm14GQVUOysu5f5R .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Rm14GQVUOysu5f5R .rough-node .label text,#mermaid-svg-Rm14GQVUOysu5f5R .node .label text,#mermaid-svg-Rm14GQVUOysu5f5R .image-shape .label,#mermaid-svg-Rm14GQVUOysu5f5R .icon-shape .label{text-anchor:middle;}#mermaid-svg-Rm14GQVUOysu5f5R .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Rm14GQVUOysu5f5R .rough-node .label,#mermaid-svg-Rm14GQVUOysu5f5R .node .label,#mermaid-svg-Rm14GQVUOysu5f5R .image-shape .label,#mermaid-svg-Rm14GQVUOysu5f5R .icon-shape .label{text-align:center;}#mermaid-svg-Rm14GQVUOysu5f5R .node.clickable{cursor:pointer;}#mermaid-svg-Rm14GQVUOysu5f5R .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Rm14GQVUOysu5f5R .arrowheadPath{fill:#333333;}#mermaid-svg-Rm14GQVUOysu5f5R .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Rm14GQVUOysu5f5R .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Rm14GQVUOysu5f5R .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Rm14GQVUOysu5f5R .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Rm14GQVUOysu5f5R .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Rm14GQVUOysu5f5R .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Rm14GQVUOysu5f5R .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Rm14GQVUOysu5f5R .cluster text{fill:#333;}#mermaid-svg-Rm14GQVUOysu5f5R .cluster span{color:#333;}#mermaid-svg-Rm14GQVUOysu5f5R div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Rm14GQVUOysu5f5R .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Rm14GQVUOysu5f5R rect.text{fill:none;stroke-width:0;}#mermaid-svg-Rm14GQVUOysu5f5R .icon-shape,#mermaid-svg-Rm14GQVUOysu5f5R .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Rm14GQVUOysu5f5R .icon-shape p,#mermaid-svg-Rm14GQVUOysu5f5R .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Rm14GQVUOysu5f5R .icon-shape .label rect,#mermaid-svg-Rm14GQVUOysu5f5R .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Rm14GQVUOysu5f5R .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Rm14GQVUOysu5f5R .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Rm14GQVUOysu5f5R :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 用户 Controller
WorkQueue (本次分析)
Informer Pipeline
queue.Add(key)
AddRateLimited → AddAfter
延迟到期 → Add
Get() → key
失败: queue.AddRateLimited(key)
成功: queue.Forget(key)
kube-apiserver
Reflector
DeltaFIFO
Controller
Store
sharedProcessor
TypedT
(基础去重队列)
delayingTypeT
(延迟队列)
rateLimitingTypeT
(限流队列)
EventHandler.OnAdd/OnUpdate/OnDelete
Worker 协程
for { key := queue.Get() ... queue.Done(key) }
调谐逻辑
计算期望状态 → API Server
三层嵌套:RateLimitingQueue → DelayingQueue → Queue。每一层添加一个能力维度。
1.3 数据流入流出
输入:
- EventHandler 调用
queue.Add(key)--- 将资源 key(如default/nginx-pod)入队 - 调谐失败调用
queue.AddRateLimited(key)--- 带延迟的重新入队 - 调谐成功调用
queue.Forget(key)--- 清除限流计数
输出:
- Worker 调用
queue.Get()--- 取出 key 进行调谐 - Worker 调用
queue.Done(key)--- 标记处理完成
#mermaid-svg-V9IEf2Xx1vuAExch{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-V9IEf2Xx1vuAExch .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-V9IEf2Xx1vuAExch .error-icon{fill:#552222;}#mermaid-svg-V9IEf2Xx1vuAExch .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-V9IEf2Xx1vuAExch .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-V9IEf2Xx1vuAExch .marker{fill:#333333;stroke:#333333;}#mermaid-svg-V9IEf2Xx1vuAExch .marker.cross{stroke:#333333;}#mermaid-svg-V9IEf2Xx1vuAExch svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-V9IEf2Xx1vuAExch p{margin:0;}#mermaid-svg-V9IEf2Xx1vuAExch .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-V9IEf2Xx1vuAExch .cluster-label text{fill:#333;}#mermaid-svg-V9IEf2Xx1vuAExch .cluster-label span{color:#333;}#mermaid-svg-V9IEf2Xx1vuAExch .cluster-label span p{background-color:transparent;}#mermaid-svg-V9IEf2Xx1vuAExch .label text,#mermaid-svg-V9IEf2Xx1vuAExch span{fill:#333;color:#333;}#mermaid-svg-V9IEf2Xx1vuAExch .node rect,#mermaid-svg-V9IEf2Xx1vuAExch .node circle,#mermaid-svg-V9IEf2Xx1vuAExch .node ellipse,#mermaid-svg-V9IEf2Xx1vuAExch .node polygon,#mermaid-svg-V9IEf2Xx1vuAExch .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-V9IEf2Xx1vuAExch .rough-node .label text,#mermaid-svg-V9IEf2Xx1vuAExch .node .label text,#mermaid-svg-V9IEf2Xx1vuAExch .image-shape .label,#mermaid-svg-V9IEf2Xx1vuAExch .icon-shape .label{text-anchor:middle;}#mermaid-svg-V9IEf2Xx1vuAExch .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-V9IEf2Xx1vuAExch .rough-node .label,#mermaid-svg-V9IEf2Xx1vuAExch .node .label,#mermaid-svg-V9IEf2Xx1vuAExch .image-shape .label,#mermaid-svg-V9IEf2Xx1vuAExch .icon-shape .label{text-align:center;}#mermaid-svg-V9IEf2Xx1vuAExch .node.clickable{cursor:pointer;}#mermaid-svg-V9IEf2Xx1vuAExch .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-V9IEf2Xx1vuAExch .arrowheadPath{fill:#333333;}#mermaid-svg-V9IEf2Xx1vuAExch .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-V9IEf2Xx1vuAExch .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-V9IEf2Xx1vuAExch .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-V9IEf2Xx1vuAExch .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-V9IEf2Xx1vuAExch .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-V9IEf2Xx1vuAExch .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-V9IEf2Xx1vuAExch .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-V9IEf2Xx1vuAExch .cluster text{fill:#333;}#mermaid-svg-V9IEf2Xx1vuAExch .cluster span{color:#333;}#mermaid-svg-V9IEf2Xx1vuAExch div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-V9IEf2Xx1vuAExch .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-V9IEf2Xx1vuAExch rect.text{fill:none;stroke-width:0;}#mermaid-svg-V9IEf2Xx1vuAExch .icon-shape,#mermaid-svg-V9IEf2Xx1vuAExch .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-V9IEf2Xx1vuAExch .icon-shape p,#mermaid-svg-V9IEf2Xx1vuAExch .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-V9IEf2Xx1vuAExch .icon-shape .label rect,#mermaid-svg-V9IEf2Xx1vuAExch .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-V9IEf2Xx1vuAExch .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-V9IEf2Xx1vuAExch .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-V9IEf2Xx1vuAExch :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 出队端
WorkQueue
入队端
Add(key)
AddRateLimited(key)
AddAfter(key, duration)
Queue (去重)
DelayingQueue (延迟)
RateLimitingQueue (限流)
Get() → (key, shutdown)
Done(key)
Forget(key)
二、模块整体结构
2.1 接口层次与类结构
渲染错误: Mermaid 渲染失败: Parse error on line 35: ... -stopCh chan struct{} -stopOnce -----------------------^ Expecting 'STRUCT_STOP', 'MEMBER', got 'OPEN_IN_STRUCT'
2.2 TypedT --- 基础队列结构体字段详解
go
type Typed[T comparable] struct {
// ─── 核心:三集合模型 ───
// queue: 有序队列,决定处理顺序(FIFO)
// 不变量:queue 中的每个元素一定在 dirty 中,且不在 processing 中
queue Queue[T]
// dirty: 需要处理的 item 集合
// 包含:queue 中的 + 正在处理但又被标记为需要重新处理的
// 作用:去重,同一 item 不会在 queue 中出现两次
dirty sets.Set[T]
// processing: 正在处理的 item 集合
// 与 dirty 可能重叠:item 正在处理但又被 Add → 留在 dirty,
// 等 Done 时检查 dirty 再重新入 queue
processing sets.Set[T]
// ─── 同步原语 ───
// cond: 条件变量,用于 Get() 阻塞等待
// 绑定 mutex: cond.L 是队列的互斥锁
// Signal: Add/Done 时通知等待的 Get
// Broadcast: ShutDown 时通知所有等待的 Get
cond *sync.Cond
// ─── 关闭状态 ───
// shuttingDown: 是否正在关闭
shuttingDown bool
// drain: 是否等待所有 item 处理完成再关闭
// ShutDown: drain=false, 立即停止
// ShutDownWithDrain: drain=true, 等待 processing 清空
drain bool
// ─── 指标与配置 ───
metrics queueMetrics[T]
unfinishedWorkUpdatePeriod time.Duration
clock clock.WithTicker
// ─── 生命周期管理 ───
// wg: 等待后台 goroutine 退出
wg sync.WaitGroup
// stopCh: 通知后台 goroutine 退出
stopCh chan struct{}
// stopOnce: 确保 stopCh 只关闭一次
stopOnce sync.Once
}
三集合模型核心不变量
#mermaid-svg-iYMXaMf49fRddbYh{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-iYMXaMf49fRddbYh .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-iYMXaMf49fRddbYh .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-iYMXaMf49fRddbYh .error-icon{fill:#552222;}#mermaid-svg-iYMXaMf49fRddbYh .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-iYMXaMf49fRddbYh .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-iYMXaMf49fRddbYh .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-iYMXaMf49fRddbYh .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-iYMXaMf49fRddbYh .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-iYMXaMf49fRddbYh .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-iYMXaMf49fRddbYh .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-iYMXaMf49fRddbYh .marker{fill:#333333;stroke:#333333;}#mermaid-svg-iYMXaMf49fRddbYh .marker.cross{stroke:#333333;}#mermaid-svg-iYMXaMf49fRddbYh svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-iYMXaMf49fRddbYh p{margin:0;}#mermaid-svg-iYMXaMf49fRddbYh .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-iYMXaMf49fRddbYh .cluster-label text{fill:#333;}#mermaid-svg-iYMXaMf49fRddbYh .cluster-label span{color:#333;}#mermaid-svg-iYMXaMf49fRddbYh .cluster-label span p{background-color:transparent;}#mermaid-svg-iYMXaMf49fRddbYh .label text,#mermaid-svg-iYMXaMf49fRddbYh span{fill:#333;color:#333;}#mermaid-svg-iYMXaMf49fRddbYh .node rect,#mermaid-svg-iYMXaMf49fRddbYh .node circle,#mermaid-svg-iYMXaMf49fRddbYh .node ellipse,#mermaid-svg-iYMXaMf49fRddbYh .node polygon,#mermaid-svg-iYMXaMf49fRddbYh .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-iYMXaMf49fRddbYh .rough-node .label text,#mermaid-svg-iYMXaMf49fRddbYh .node .label text,#mermaid-svg-iYMXaMf49fRddbYh .image-shape .label,#mermaid-svg-iYMXaMf49fRddbYh .icon-shape .label{text-anchor:middle;}#mermaid-svg-iYMXaMf49fRddbYh .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-iYMXaMf49fRddbYh .rough-node .label,#mermaid-svg-iYMXaMf49fRddbYh .node .label,#mermaid-svg-iYMXaMf49fRddbYh .image-shape .label,#mermaid-svg-iYMXaMf49fRddbYh .icon-shape .label{text-align:center;}#mermaid-svg-iYMXaMf49fRddbYh .node.clickable{cursor:pointer;}#mermaid-svg-iYMXaMf49fRddbYh .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-iYMXaMf49fRddbYh .arrowheadPath{fill:#333333;}#mermaid-svg-iYMXaMf49fRddbYh .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-iYMXaMf49fRddbYh .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-iYMXaMf49fRddbYh .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-iYMXaMf49fRddbYh .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-iYMXaMf49fRddbYh .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-iYMXaMf49fRddbYh .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-iYMXaMf49fRddbYh .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-iYMXaMf49fRddbYh .cluster text{fill:#333;}#mermaid-svg-iYMXaMf49fRddbYh .cluster span{color:#333;}#mermaid-svg-iYMXaMf49fRddbYh div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-iYMXaMf49fRddbYh .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-iYMXaMf49fRddbYh rect.text{fill:none;stroke-width:0;}#mermaid-svg-iYMXaMf49fRddbYh .icon-shape,#mermaid-svg-iYMXaMf49fRddbYh .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-iYMXaMf49fRddbYh .icon-shape p,#mermaid-svg-iYMXaMf49fRddbYh .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-iYMXaMf49fRddbYh .icon-shape .label rect,#mermaid-svg-iYMXaMf49fRddbYh .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-iYMXaMf49fRddbYh .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-iYMXaMf49fRddbYh .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-iYMXaMf49fRddbYh :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 不变量
三集合模型
dirty set
(需要处理的)
queue slice
(等待处理的)
processing set
(正在处理的)
queue ⊆ dirty
(queue 中的元素一定在 dirty 中)
queue ∩ processing = ∅
(queue 中的元素不在 processing 中)
dirty ∩ processing ≠ ∅
(可重叠:正在处理又被 Add)
三集合状态转移:
| 操作 | dirty | queue | processing |
|---|---|---|---|
Add(x) (新 item) |
+x | +x | --- |
Add(x) (已在 dirty,不在 processing) |
不变 | 不变 (Touch) | --- |
Add(x) (已在 dirty,也在 processing) |
不变 | 不变 | --- |
Add(x) (不在 dirty,在 processing) |
+x | 不变 | --- |
Get() → x |
-x | -x (Pop) | +x |
Done(x) (dirty 中无 x) |
--- | --- | -x |
Done(x) (dirty 中有 x) |
--- | +x (重新入队) | -x |
2.3 delayingTypeT --- 延迟队列字段详解
go
type delayingType[T comparable] struct {
TypedInterface[T] // 嵌入基础队列,继承 Add/Get/Done/ShutDown
clock clock.Clock // 时钟,可注入 FakeClock 测试
stopCh chan struct{} // 通知 waitingLoop 退出
stopOnce sync.Once // 确保 stopCh 只关闭一次
heartbeat clock.Ticker // 心跳定时器(10s),保险机制
// waitingForAddCh: 接收外部 AddAfter 请求的缓冲通道
// 容量 1000,避免 AddAfter 阻塞调用方
waitingForAddCh chan *waitFor[T]
metrics retryMetrics // 重试指标
}
waitFor --- 延迟条目
go
type waitFor[T any] struct {
data T // 待入队的 item
readyAt time.Time // 应该入队的时间
index int // 在优先堆中的索引,heap.Fix 使用
}
waitForPriorityQueue --- 最小堆
go
type waitForPriorityQueue[T any] []*waitFor[T]
// 实现 heap.Interface
// 按 readyAt 排序,最近到期的在堆顶(index 0)
// 支持 insert 去重:同一 data 只保留最早 readyAt
2.4 rateLimitingTypeT --- 限流队列字段详解
go
type rateLimitingType[T comparable] struct {
TypedDelayingInterface[T] // 嵌入延迟队列
rateLimiter TypedRateLimiter[T] // 限流器,决定 When(item) 的延迟
}
2.5 核心方法清单
TypedT 基础队列方法
| 方法 | 作用 | 阻塞 |
|---|---|---|
Add(item) |
标记 item 需要 processing | ❌ |
Len() |
返回 queue 长度 | ❌ |
Get() |
取出下一个 item,阻塞等待 | ✅ |
Done(item) |
标记 item 处理完成 | ❌ |
ShutDown() |
关闭队列,不等待处理完成 | ❌ (等 wg) |
ShutDownWithDrain() |
关闭队列,等待处理完成 | ✅ |
ShuttingDown() |
是否正在关闭 | ❌ |
delayingTypeT 延迟队列方法
| 方法 | 作用 |
|---|---|
AddAfter(item, duration) |
延迟 duration 后入队 |
ShutDown() |
关闭延迟队列 |
waitingLoop() |
后台协程:优先堆 + 定时器调度 |
rateLimitingTypeT 限流队列方法
| 方法 | 作用 |
|---|---|
AddRateLimited(item) |
通过限流器计算延迟后 AddAfter |
Forget(item) |
清除 item 的失败计数 |
NumRequeues(item) |
返回 item 的重试次数 |
2.6 内部调用关系
#mermaid-svg-dYIwtVFl7bZRGsf2{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-dYIwtVFl7bZRGsf2 .error-icon{fill:#552222;}#mermaid-svg-dYIwtVFl7bZRGsf2 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-dYIwtVFl7bZRGsf2 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .marker.cross{stroke:#333333;}#mermaid-svg-dYIwtVFl7bZRGsf2 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-dYIwtVFl7bZRGsf2 p{margin:0;}#mermaid-svg-dYIwtVFl7bZRGsf2 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster-label text{fill:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster-label span{color:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster-label span p{background-color:transparent;}#mermaid-svg-dYIwtVFl7bZRGsf2 .label text,#mermaid-svg-dYIwtVFl7bZRGsf2 span{fill:#333;color:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .node rect,#mermaid-svg-dYIwtVFl7bZRGsf2 .node circle,#mermaid-svg-dYIwtVFl7bZRGsf2 .node ellipse,#mermaid-svg-dYIwtVFl7bZRGsf2 .node polygon,#mermaid-svg-dYIwtVFl7bZRGsf2 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .rough-node .label text,#mermaid-svg-dYIwtVFl7bZRGsf2 .node .label text,#mermaid-svg-dYIwtVFl7bZRGsf2 .image-shape .label,#mermaid-svg-dYIwtVFl7bZRGsf2 .icon-shape .label{text-anchor:middle;}#mermaid-svg-dYIwtVFl7bZRGsf2 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .rough-node .label,#mermaid-svg-dYIwtVFl7bZRGsf2 .node .label,#mermaid-svg-dYIwtVFl7bZRGsf2 .image-shape .label,#mermaid-svg-dYIwtVFl7bZRGsf2 .icon-shape .label{text-align:center;}#mermaid-svg-dYIwtVFl7bZRGsf2 .node.clickable{cursor:pointer;}#mermaid-svg-dYIwtVFl7bZRGsf2 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .arrowheadPath{fill:#333333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-dYIwtVFl7bZRGsf2 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-dYIwtVFl7bZRGsf2 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-dYIwtVFl7bZRGsf2 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster text{fill:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 .cluster span{color:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-dYIwtVFl7bZRGsf2 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-dYIwtVFl7bZRGsf2 rect.text{fill:none;stroke-width:0;}#mermaid-svg-dYIwtVFl7bZRGsf2 .icon-shape,#mermaid-svg-dYIwtVFl7bZRGsf2 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-dYIwtVFl7bZRGsf2 .icon-shape p,#mermaid-svg-dYIwtVFl7bZRGsf2 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-dYIwtVFl7bZRGsf2 .icon-shape .label rect,#mermaid-svg-dYIwtVFl7bZRGsf2 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-dYIwtVFl7bZRGsf2 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-dYIwtVFl7bZRGsf2 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-dYIwtVFl7bZRGsf2 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} RateLimiter 实现
Queue (最内层)
DelayingQueue (中间层)
RateLimitingQueue (最外层)
duration>0
duration≤0
DefaultControllerRateLimiter
AddRateLimited(item)
Forget(item)
NumRequeues(item)
rateLimiter.When(item) → duration
AddAfter(item, duration)
waitingLoop() (后台协程)
waitForPriorityQueue (最小堆)
insert() 去重+更新
到期 → Add(item)
Add(item)
Get()
Done(item)
ShutDown()
ShutDownWithDrain()
ItemExponentialFailureRateLimiter
baseDelay×2^failures
BucketRateLimiter
10 QPS, burst 100
MaxOfRateLimiter
max(exp, bucket)
ItemFastSlowRateLimiter
WithMaxWaitRateLimiter
三、核心业务逻辑深度解析 --- 基础队列 (queue.go)
3.1 Add --- 入队逻辑
go
func (q *Typed[T]) Add(item T) {
// ─── 步骤1: 加锁 ───
q.cond.L.Lock()
defer q.cond.L.Unlock()
// ─── 步骤2: 关闭检查 ───
if q.shuttingDown {
return
// 队列已关闭 → 静默忽略新 item
// 不报错,不 panic,这是有意的设计
}
// ─── 步骤3: 去重检查 ───
if q.dirty.Has(item) {
// item 已在 dirty 集合中(需要处理但尚未处理完)
if !q.processing.Has(item) {
// item 在 dirty 但不在 processing
// 说明 item 在 queue 中等待,尚未被 Get
// 调用 Touch:允许优先队列实现更新优先级
q.queue.Touch(item)
}
// 如果 item 同时在 processing → 什么都不做
// 因为 processing + dirty 意味着处理完后会自动重新入队
return
}
// ─── 步骤4: 新 item ───
q.metrics.add(item) // 指标:adds++ + depth++
q.dirty.Insert(item) // 标记为需要处理
if q.processing.Has(item) {
// item 正在被处理 → 只加入 dirty,不加入 queue
// Done 时会检查 dirty,自动重新入队
return
}
// ─── 步骤5: 入队 + 唤醒 ───
q.queue.Push(item) // 实际入队
q.cond.Signal() // 唤醒一个阻塞的 Get
}
Add 的四象限决策:
#mermaid-svg-FUTiHNfuZ37P4qKO{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-FUTiHNfuZ37P4qKO .error-icon{fill:#552222;}#mermaid-svg-FUTiHNfuZ37P4qKO .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-FUTiHNfuZ37P4qKO .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-FUTiHNfuZ37P4qKO .marker{fill:#333333;stroke:#333333;}#mermaid-svg-FUTiHNfuZ37P4qKO .marker.cross{stroke:#333333;}#mermaid-svg-FUTiHNfuZ37P4qKO svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-FUTiHNfuZ37P4qKO p{margin:0;}#mermaid-svg-FUTiHNfuZ37P4qKO .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster-label text{fill:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster-label span{color:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster-label span p{background-color:transparent;}#mermaid-svg-FUTiHNfuZ37P4qKO .label text,#mermaid-svg-FUTiHNfuZ37P4qKO span{fill:#333;color:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO .node rect,#mermaid-svg-FUTiHNfuZ37P4qKO .node circle,#mermaid-svg-FUTiHNfuZ37P4qKO .node ellipse,#mermaid-svg-FUTiHNfuZ37P4qKO .node polygon,#mermaid-svg-FUTiHNfuZ37P4qKO .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-FUTiHNfuZ37P4qKO .rough-node .label text,#mermaid-svg-FUTiHNfuZ37P4qKO .node .label text,#mermaid-svg-FUTiHNfuZ37P4qKO .image-shape .label,#mermaid-svg-FUTiHNfuZ37P4qKO .icon-shape .label{text-anchor:middle;}#mermaid-svg-FUTiHNfuZ37P4qKO .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-FUTiHNfuZ37P4qKO .rough-node .label,#mermaid-svg-FUTiHNfuZ37P4qKO .node .label,#mermaid-svg-FUTiHNfuZ37P4qKO .image-shape .label,#mermaid-svg-FUTiHNfuZ37P4qKO .icon-shape .label{text-align:center;}#mermaid-svg-FUTiHNfuZ37P4qKO .node.clickable{cursor:pointer;}#mermaid-svg-FUTiHNfuZ37P4qKO .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-FUTiHNfuZ37P4qKO .arrowheadPath{fill:#333333;}#mermaid-svg-FUTiHNfuZ37P4qKO .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-FUTiHNfuZ37P4qKO .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-FUTiHNfuZ37P4qKO .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-FUTiHNfuZ37P4qKO .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-FUTiHNfuZ37P4qKO .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-FUTiHNfuZ37P4qKO .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster text{fill:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO .cluster span{color:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-FUTiHNfuZ37P4qKO .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-FUTiHNfuZ37P4qKO rect.text{fill:none;stroke-width:0;}#mermaid-svg-FUTiHNfuZ37P4qKO .icon-shape,#mermaid-svg-FUTiHNfuZ37P4qKO .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-FUTiHNfuZ37P4qKO .icon-shape p,#mermaid-svg-FUTiHNfuZ37P4qKO .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-FUTiHNfuZ37P4qKO .icon-shape .label rect,#mermaid-svg-FUTiHNfuZ37P4qKO .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-FUTiHNfuZ37P4qKO .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-FUTiHNfuZ37P4qKO .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-FUTiHNfuZ37P4qKO :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
在 dirty, 不在 processing
在 dirty, 在 processing
不在 dirty, 不在 processing
不在 dirty, 在 processing
Add(item)
dirty.Has(item)?
processing.Has(item)?
新 item:
dirty.Insert + queue.Push + Signal
Touch(item)
(更新优先级)
只 dirty.Insert
(Done 时会重新入队)
3.2 Get --- 出队逻辑
go
func (q *Typed[T]) Get() (item T, shutdown bool) {
// ─── 步骤1: 加锁 ───
q.cond.L.Lock()
defer q.cond.L.Unlock()
// ─── 步骤2: 阻塞等待 ───
for q.queue.Len() == 0 && !q.shuttingDown {
q.cond.Wait()
// 释放锁 → 阻塞 → 被 Signal/Broadcast 唤醒 → 重新获取锁
// 被唤醒的场景:
// 1. Add 入队 → cond.Signal()
// 2. Done 且 dirty 有 item → cond.Signal()
// 3. ShutDown → cond.Broadcast()
}
// ─── 步骤3: 关闭检查 ───
if q.queue.Len() == 0 {
// 退出 for 循环但 queue 为空 → 说明 shuttingDown=true
return *new(T), true
// 返回零值 + shutdown=true
// 调用方应退出 worker 协程
}
// ─── 步骤4: 出队 ───
item = q.queue.Pop() // FIFO 出队
q.metrics.get(item) // 指标:depth-- + 记录 latency
q.processing.Insert(item) // 标记为正在处理
q.dirty.Delete(item) // 从 dirty 移除
// 注意:如果 Add 在 processing 期间又标记了 dirty
// 此时 dirty 中仍有 item,但已经从 queue 移除了
// 等 Done 时会检查 dirty,重新入队
return item, false
}
Get 的状态转移:
processing set dirty set queue slice TypedT Worker 协程 processing set dirty set queue slice TypedT Worker 协程 #mermaid-svg-b52uVW3khQZRmwDK{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-b52uVW3khQZRmwDK .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-b52uVW3khQZRmwDK .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-b52uVW3khQZRmwDK .error-icon{fill:#552222;}#mermaid-svg-b52uVW3khQZRmwDK .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-b52uVW3khQZRmwDK .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-b52uVW3khQZRmwDK .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-b52uVW3khQZRmwDK .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-b52uVW3khQZRmwDK .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-b52uVW3khQZRmwDK .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-b52uVW3khQZRmwDK .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-b52uVW3khQZRmwDK .marker{fill:#333333;stroke:#333333;}#mermaid-svg-b52uVW3khQZRmwDK .marker.cross{stroke:#333333;}#mermaid-svg-b52uVW3khQZRmwDK svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-b52uVW3khQZRmwDK p{margin:0;}#mermaid-svg-b52uVW3khQZRmwDK .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-b52uVW3khQZRmwDK text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-b52uVW3khQZRmwDK .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-b52uVW3khQZRmwDK .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-b52uVW3khQZRmwDK .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-b52uVW3khQZRmwDK .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-b52uVW3khQZRmwDK #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-b52uVW3khQZRmwDK .sequenceNumber{fill:white;}#mermaid-svg-b52uVW3khQZRmwDK #sequencenumber{fill:#333;}#mermaid-svg-b52uVW3khQZRmwDK #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-b52uVW3khQZRmwDK .messageText{fill:#333;stroke:none;}#mermaid-svg-b52uVW3khQZRmwDK .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-b52uVW3khQZRmwDK .labelText,#mermaid-svg-b52uVW3khQZRmwDK .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-b52uVW3khQZRmwDK .loopText,#mermaid-svg-b52uVW3khQZRmwDK .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-b52uVW3khQZRmwDK .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-b52uVW3khQZRmwDK .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-b52uVW3khQZRmwDK .noteText,#mermaid-svg-b52uVW3khQZRmwDK .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-b52uVW3khQZRmwDK .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-b52uVW3khQZRmwDK .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-b52uVW3khQZRmwDK .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-b52uVW3khQZRmwDK .actorPopupMenu{position:absolute;}#mermaid-svg-b52uVW3khQZRmwDK .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-b52uVW3khQZRmwDK .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-b52uVW3khQZRmwDK .actor-man circle,#mermaid-svg-b52uVW3khQZRmwDK line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-b52uVW3khQZRmwDK :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 阻塞等待... Get()cond.L.Lock()for queue.Len()==0 && !shuttingDown { Wait() }被 Signal 唤醒queue.Pop() → itemprocessing.Insert(item)dirty.Delete(item)cond.L.Unlock()(item, false)
3.3 Done --- 完成逻辑
go
func (q *Typed[T]) Done(item T) {
// ─── 步骤1: 加锁 ───
q.cond.L.Lock()
defer q.cond.L.Unlock()
// ─── 步骤2: 指标 ───
q.metrics.done(item) // 记录 workDuration
// ─── 步骤3: 从 processing 移除 ───
q.processing.Delete(item)
// ─── 步骤4: 重新入队检查 ───
if q.dirty.Has(item) {
// 处理期间又被 Add 标记为需要重新处理
q.queue.Push(item) // 重新入队
q.cond.Signal() // 唤醒 Get
} else if q.processing.Len() == 0 {
// processing 清空 且 dirty 无此 item
// 唤醒 ShutDownWithDrain 的等待
// 条件:q.processing.Len() == 0
q.cond.Signal()
}
// 如果 dirty 无此 item 且 processing 不为空 → 不 Signal
// 等最后一个 item Done 时自然会 Signal
}
Done 的关键设计 :处理期间被 Add → 不立即入 queue,而是等 Done 时再入。这保证了同一 item 不会被并发处理。
3.4 ShutDown / ShutDownWithDrain --- 关闭逻辑
go
// ShutDown: 立即关闭,不等待处理完成
func (q *Typed[T]) ShutDown() {
defer q.wg.Wait() // 等待后台 goroutine 退出
q.stopOnce.Do(func() {
defer close(q.stopCh) // 通知 updateUnfinishedWorkLoop 退出
})
q.cond.L.Lock()
defer q.cond.L.Unlock()
q.drain = false // 不等待 drain
q.shuttingDown = true // 标记关闭
q.cond.Broadcast() // 唤醒所有阻塞的 Get
// Get 检测到 shuttingDown 且 queue 为空 → 返回 shutdown=true
}
// ShutDownWithDrain: 等待所有 item 处理完成
func (q *Typed[T]) ShutDownWithDrain() {
defer q.wg.Wait()
q.stopOnce.Do(func() {
defer close(q.stopCh)
})
q.cond.L.Lock()
defer q.cond.L.Unlock()
q.drain = true // 标记等待 drain
q.shuttingDown = true // 标记关闭
q.cond.Broadcast() // 唤醒所有阻塞的 Get
// 等待 processing 清空
for q.processing.Len() != 0 && q.drain {
q.cond.Wait()
// 每个 Done 检查 processing.Len()==0 时会 Signal
// 或者 ShutDown 被再次调用(drain=false)→ 也会退出
}
}
关闭流程:
后台 goroutine Worker 协程 TypedT 调用方 后台 goroutine Worker 协程 TypedT 调用方 #mermaid-svg-0J9sGBoa52lI7UxC{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-0J9sGBoa52lI7UxC .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-0J9sGBoa52lI7UxC .error-icon{fill:#552222;}#mermaid-svg-0J9sGBoa52lI7UxC .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-0J9sGBoa52lI7UxC .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-0J9sGBoa52lI7UxC .marker{fill:#333333;stroke:#333333;}#mermaid-svg-0J9sGBoa52lI7UxC .marker.cross{stroke:#333333;}#mermaid-svg-0J9sGBoa52lI7UxC svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-0J9sGBoa52lI7UxC p{margin:0;}#mermaid-svg-0J9sGBoa52lI7UxC .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-0J9sGBoa52lI7UxC text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-0J9sGBoa52lI7UxC .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-0J9sGBoa52lI7UxC .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-0J9sGBoa52lI7UxC .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-0J9sGBoa52lI7UxC .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-0J9sGBoa52lI7UxC #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-0J9sGBoa52lI7UxC .sequenceNumber{fill:white;}#mermaid-svg-0J9sGBoa52lI7UxC #sequencenumber{fill:#333;}#mermaid-svg-0J9sGBoa52lI7UxC #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-0J9sGBoa52lI7UxC .messageText{fill:#333;stroke:none;}#mermaid-svg-0J9sGBoa52lI7UxC .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-0J9sGBoa52lI7UxC .labelText,#mermaid-svg-0J9sGBoa52lI7UxC .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-0J9sGBoa52lI7UxC .loopText,#mermaid-svg-0J9sGBoa52lI7UxC .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-0J9sGBoa52lI7UxC .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-0J9sGBoa52lI7UxC .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-0J9sGBoa52lI7UxC .noteText,#mermaid-svg-0J9sGBoa52lI7UxC .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-0J9sGBoa52lI7UxC .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-0J9sGBoa52lI7UxC .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-0J9sGBoa52lI7UxC .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-0J9sGBoa52lI7UxC .actorPopupMenu{position:absolute;}#mermaid-svg-0J9sGBoa52lI7UxC .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-0J9sGBoa52lI7UxC .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-0J9sGBoa52lI7UxC .actor-man circle,#mermaid-svg-0J9sGBoa52lI7UxC line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-0J9sGBoa52lI7UxC :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Get() 发现 shuttingDown+queue为空 → 返回 shutdown=true loop等待 processing 清空 processing 清空 → 退出 for 循环 ShutDownWithDrain()shuttingDown=true, drain=truecond.Broadcast()Done(item)processing.Delete(item)processing.Len()==0? → Signalclose(stopCh)wg.Wait()返回
3.5 updateUnfinishedWorkLoop --- 后台指标更新
go
func (q *Typed[T]) updateUnfinishedWorkLoop() {
t := q.clock.NewTicker(q.unfinishedWorkUpdatePeriod)
// 默认 500ms 刷新一次
defer t.Stop()
for {
select {
case <-t.C():
q.updateUnfinishedWork()
// 更新 unfinishedWorkSeconds + longestRunningProcessor
case <-q.stopCh:
return
// 队列关闭 → 退出
}
}
}
3.6 构造函数体系
go
func newQueueWithConfig[T comparable](config TypedQueueConfig[T], updatePeriod time.Duration) *Typed[T] {
metricsProvider := globalMetricsProvider
if config.MetricsProvider != nil {
metricsProvider = config.MetricsProvider
}
if config.Clock == nil {
config.Clock = clock.RealClock{}
}
if config.Queue == nil {
config.Queue = DefaultQueue[T]() // 默认 slice FIFO
}
return newQueue(
config.Clock,
config.Queue,
newQueueMetrics[T](metricsProvider, config.Name, config.Clock),
updatePeriod,
)
}
func newQueue[T comparable](c clock.WithTicker, queue Queue[T], metrics queueMetrics[T], updatePeriod time.Duration) *Typed[T] {
t := &Typed[T]{
clock: c,
queue: queue,
dirty: sets.Set[T]{},
processing: sets.Set[T]{},
cond: sync.NewCond(&sync.Mutex{}),
metrics: metrics,
unfinishedWorkUpdatePeriod: updatePeriod,
stopCh: make(chan struct{}),
}
// 如果有 metrics → 启动后台 goroutine
if _, ok := metrics.(noMetrics[T]); !ok {
t.wg.Go(t.updateUnfinishedWorkLoop)
}
return t
}
四、核心业务逻辑深度解析 --- 延迟队列 (delaying_queue.go)
4.1 AddAfter --- 延迟入队
go
func (q *delayingType[T]) AddAfter(item T, duration time.Duration) {
// ─── 步骤1: 关闭检查 ───
if q.ShuttingDown() {
return
// 队列关闭 → 忽略
}
// ─── 步骤2: 指标 ───
q.metrics.retry()
// 每次调用 AddAfter 都计为一次 retry
// ─── 步骤3: 零延迟直接入队 ───
if duration <= 0 {
q.Add(item)
return
// 无需延迟 → 走基础队列的 Add
}
// ─── 步骤4: 发送到 waitingForAddCh ───
select {
case <-q.stopCh:
// 队列关闭 → 不入队
case q.waitingForAddCh <- &waitFor[T]{
data: item,
readyAt: q.clock.Now().Add(duration),
// readyAt = 当前时间 + 延迟时间
}:
// 成功写入缓冲通道
// waitingLoop 会消费这个通道
}
}
4.2 waitingLoop --- 延迟调度核心
go
func (q *delayingType[T]) waitingLoop(logger klog.Logger) {
defer utilruntime.HandleCrashWithLogger(logger)
// panic 恢复
// ─── 初始化 ───
never := make(<-chan time.Time)
// 永远不会触发的 channel,当堆为空时使用
var nextReadyAtTimer clock.Timer
// 下一个到期定时器
waitingForQueue := &waitForPriorityQueue[T]{}
heap.Init(waitingForQueue)
// 最小堆,按 readyAt 排序
waitingEntryByData := map[T]*waitFor[T]{}
// 按 data 去重的索引,支持更新 readyAt
// ─── 主循环 ───
for {
// 关闭检查
if q.TypedInterface.ShuttingDown() {
return
}
now := q.clock.Now()
// ─── 处理已到期的条目 ───
for waitingForQueue.Len() > 0 {
entry := waitingForQueue.Peek().(*waitFor[T])
if entry.readyAt.After(now) {
break
// 堆顶未到期 → 跳出
}
entry = heap.Pop(waitingForQueue).(*waitFor[T])
q.Add(entry.data)
// 到期 → 直接 Add 到基础队列
delete(waitingEntryByData, entry.data)
// 从去重索引删除
}
// ─── 设置下一个定时器 ───
nextReadyAt := never
if waitingForQueue.Len() > 0 {
if nextReadyAtTimer != nil {
nextReadyAtTimer.Stop()
// 停止旧的定时器
}
entry := waitingForQueue.Peek().(*waitFor[T])
nextReadyAtTimer = q.clock.NewTimer(entry.readyAt.Sub(now))
nextReadyAt = nextReadyAtTimer.C()
// 堆顶条目的剩余时间
}
// ─── 多路复用等待 ───
select {
case <-q.stopCh:
return
// 关闭信号 → 退出
case <-q.heartbeat.C():
// 心跳(10s),保险机制
// 即使定时器出问题,最多延迟 10s
// continue → 重新检查堆顶
case <-nextReadyAt:
// 堆顶条目到期
// continue → 重新检查堆顶并处理
case waitEntry := <-q.waitingForAddCh:
// 接收到新的延迟条目
if waitEntry.readyAt.After(q.clock.Now()) {
insert(waitingForQueue, waitingEntryByData, waitEntry)
// 未到期 → 插入堆
} else {
q.Add(waitEntry.data)
// 已到期 → 直接 Add
}
// 尽量排空通道(非阻塞)
drained := false
for !drained {
select {
case waitEntry := <-q.waitingForAddCh:
if waitEntry.readyAt.After(q.clock.Now()) {
insert(waitingForQueue, waitingEntryByData, waitEntry)
} else {
q.Add(waitEntry.data)
}
default:
drained = true
}
}
}
}
}
waitingLoop 完整流程:
#mermaid-svg-NKxSHO02JP9WWX2v{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NKxSHO02JP9WWX2v .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NKxSHO02JP9WWX2v .error-icon{fill:#552222;}#mermaid-svg-NKxSHO02JP9WWX2v .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NKxSHO02JP9WWX2v .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NKxSHO02JP9WWX2v .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NKxSHO02JP9WWX2v .marker.cross{stroke:#333333;}#mermaid-svg-NKxSHO02JP9WWX2v svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NKxSHO02JP9WWX2v p{margin:0;}#mermaid-svg-NKxSHO02JP9WWX2v .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-NKxSHO02JP9WWX2v .cluster-label text{fill:#333;}#mermaid-svg-NKxSHO02JP9WWX2v .cluster-label span{color:#333;}#mermaid-svg-NKxSHO02JP9WWX2v .cluster-label span p{background-color:transparent;}#mermaid-svg-NKxSHO02JP9WWX2v .label text,#mermaid-svg-NKxSHO02JP9WWX2v span{fill:#333;color:#333;}#mermaid-svg-NKxSHO02JP9WWX2v .node rect,#mermaid-svg-NKxSHO02JP9WWX2v .node circle,#mermaid-svg-NKxSHO02JP9WWX2v .node ellipse,#mermaid-svg-NKxSHO02JP9WWX2v .node polygon,#mermaid-svg-NKxSHO02JP9WWX2v .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NKxSHO02JP9WWX2v .rough-node .label text,#mermaid-svg-NKxSHO02JP9WWX2v .node .label text,#mermaid-svg-NKxSHO02JP9WWX2v .image-shape .label,#mermaid-svg-NKxSHO02JP9WWX2v .icon-shape .label{text-anchor:middle;}#mermaid-svg-NKxSHO02JP9WWX2v .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NKxSHO02JP9WWX2v .rough-node .label,#mermaid-svg-NKxSHO02JP9WWX2v .node .label,#mermaid-svg-NKxSHO02JP9WWX2v .image-shape .label,#mermaid-svg-NKxSHO02JP9WWX2v .icon-shape .label{text-align:center;}#mermaid-svg-NKxSHO02JP9WWX2v .node.clickable{cursor:pointer;}#mermaid-svg-NKxSHO02JP9WWX2v .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NKxSHO02JP9WWX2v .arrowheadPath{fill:#333333;}#mermaid-svg-NKxSHO02JP9WWX2v .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NKxSHO02JP9WWX2v .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NKxSHO02JP9WWX2v .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NKxSHO02JP9WWX2v .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NKxSHO02JP9WWX2v .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NKxSHO02JP9WWX2v .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NKxSHO02JP9WWX2v .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NKxSHO02JP9WWX2v .cluster text{fill:#333;}#mermaid-svg-NKxSHO02JP9WWX2v .cluster span{color:#333;}#mermaid-svg-NKxSHO02JP9WWX2v div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NKxSHO02JP9WWX2v .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NKxSHO02JP9WWX2v rect.text{fill:none;stroke-width:0;}#mermaid-svg-NKxSHO02JP9WWX2v .icon-shape,#mermaid-svg-NKxSHO02JP9WWX2v .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NKxSHO02JP9WWX2v .icon-shape p,#mermaid-svg-NKxSHO02JP9WWX2v .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NKxSHO02JP9WWX2v .icon-shape .label rect,#mermaid-svg-NKxSHO02JP9WWX2v .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NKxSHO02JP9WWX2v .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NKxSHO02JP9WWX2v .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NKxSHO02JP9WWX2v :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
Yes
No
waitingLoop 启动
初始化:
waitingForQueue (最小堆)
waitingEntryByData (去重 map)
主循环
ShuttingDown?
弹出已到期条目
heap.Pop → Add(data)
设置 nextReadyAtTimer
= 堆顶 readyAt - now
select 等待
<-stopCh → return
<-heartbeat → continue
<-nextReadyAt → continue
<-waitingForAddCh
readyAt > now?
insert() 堆插入/更新
Add(data) 直接入队
排空 waitingForAddCh
4.3 insert --- 堆插入/更新
go
func insert[T comparable](q *waitForPriorityQueue[T], knownEntries map[T]*waitFor[T], entry *waitFor[T]) {
// ─── 去重:如果 item 已在堆中 ───
existing, exists := knownEntries[entry.data]
if exists {
if existing.readyAt.After(entry.readyAt) {
// 新的 readyAt 更早 → 更新
existing.readyAt = entry.readyAt
heap.Fix(q, existing.index)
// 重新调整堆,O(log n)
}
// 新的 readyAt 更晚或相等 → 忽略
return
}
// ─── 新条目:插入堆 ───
heap.Push(q, entry)
knownEntries[entry.data] = entry
}
insert 的设计:保证同一 item 在堆中只有一份,且保留最早的 readyAt。这避免了同一 item 的重复延迟入队。
4.4 优先堆实现 --- waitForPriorityQueue
go
type waitForPriorityQueue[T any] []*waitFor[T]
func (pq waitForPriorityQueue[T]) Len() int { return len(pq) }
func (pq waitForPriorityQueue[T]) Less(i, j int) bool {
return pq[i].readyAt.Before(pq[j].readyAt)
// 最小堆:readyAt 最早的在堆顶
}
func (pq waitForPriorityQueue[T]) Swap(i, j int) {
pq[i], pq[j] = pq[j], pq[i]
pq[i].index = i // 更新索引,供 heap.Fix 使用
pq[j].index = j
}
func (pq *waitForPriorityQueue[T]) Push(x interface{}) {
n := len(*pq)
item := x.(*waitFor[T])
item.index = n
*pq = append(*pq, item)
}
func (pq *waitForPriorityQueue[T]) Pop() interface{} {
n := len(*pq)
item := (*pq)[n-1]
item.index = -1 // 标记为已移除
*pq = (*pq)[0 : n-1]
return item
}
func (pq waitForPriorityQueue[T]) Peek() interface{} {
return pq[0] // 查看堆顶,不移除
}