NVIDIA k8s-device-plugin v0.19.3 核心插件模块深度分析
一、模块定位
1.1 业务职责
NVIDIA k8s-device-plugin 的核心插件模块是整个项目的心脏，负责：
- 设备发现：通过 NVML 或 Tegra 策略发现节点上的 GPU/MIG 设备
- 资源注册：将 GPU 资源注册到 Kubernetes kubelet，使其成为可调度的 Extended Resource
- 设备分配：响应 kubelet 的 Allocate 请求，为容器分配 GPU 设备
- 健康监控：持续监控 GPU 健康状态，不健康设备自动标记为 Unhealthy
- CDI 支持：生成 CDI (Container Device Interface) 规范，支持多种设备列表策略
- MPS 支持：支持 MPS (Multi-Process Service) 共享 GPU 资源
- IMEX 支持：支持 IMEX 通道注入
1.2 在系统中的位置
#mermaid-svg-t856rToyw3ngUs1s{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-t856rToyw3ngUs1s .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-t856rToyw3ngUs1s .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-t856rToyw3ngUs1s .error-icon{fill:#552222;}#mermaid-svg-t856rToyw3ngUs1s .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-t856rToyw3ngUs1s .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-t856rToyw3ngUs1s .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-t856rToyw3ngUs1s .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-t856rToyw3ngUs1s .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-t856rToyw3ngUs1s .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-t856rToyw3ngUs1s .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-t856rToyw3ngUs1s .marker{fill:#333333;stroke:#333333;}#mermaid-svg-t856rToyw3ngUs1s .marker.cross{stroke:#333333;}#mermaid-svg-t856rToyw3ngUs1s svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-t856rToyw3ngUs1s p{margin:0;}#mermaid-svg-t856rToyw3ngUs1s .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-t856rToyw3ngUs1s .cluster-label text{fill:#333;}#mermaid-svg-t856rToyw3ngUs1s .cluster-label span{color:#333;}#mermaid-svg-t856rToyw3ngUs1s .cluster-label span p{background-color:transparent;}#mermaid-svg-t856rToyw3ngUs1s .label text,#mermaid-svg-t856rToyw3ngUs1s span{fill:#333;color:#333;}#mermaid-svg-t856rToyw3ngUs1s .node rect,#mermaid-svg-t856rToyw3ngUs1s .node circle,#mermaid-svg-t856rToyw3ngUs1s .node ellipse,#mermaid-svg-t856rToyw3ngUs1s .node polygon,#mermaid-svg-t856rToyw3ngUs1s .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-t856rToyw3ngUs1s .rough-node .label text,#mermaid-svg-t856rToyw3ngUs1s .node .label text,#mermaid-svg-t856rToyw3ngUs1s .image-shape .label,#mermaid-svg-t856rToyw3ngUs1s .icon-shape .label{text-anchor:middle;}#mermaid-svg-t856rToyw3ngUs1s .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-t856rToyw3ngUs1s .rough-node .label,#mermaid-svg-t856rToyw3ngUs1s .node .label,#mermaid-svg-t856rToyw3ngUs1s .image-shape .label,#mermaid-svg-t856rToyw3ngUs1s .icon-shape .label{text-align:center;}#mermaid-svg-t856rToyw3ngUs1s .node.clickable{cursor:pointer;}#mermaid-svg-t856rToyw3ngUs1s .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-t856rToyw3ngUs1s .arrowheadPath{fill:#333333;}#mermaid-svg-t856rToyw3ngUs1s .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-t856rToyw3ngUs1s .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-t856rToyw3ngUs1s .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t856rToyw3ngUs1s .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-t856rToyw3ngUs1s .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t856rToyw3ngUs1s .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-t856rToyw3ngUs1s .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-t856rToyw3ngUs1s .cluster text{fill:#333;}#mermaid-svg-t856rToyw3ngUs1s .cluster span{color:#333;}#mermaid-svg-t856rToyw3ngUs1s div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-t856rToyw3ngUs1s .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-t856rToyw3ngUs1s rect.text{fill:none;stroke-width:0;}#mermaid-svg-t856rToyw3ngUs1s .icon-shape,#mermaid-svg-t856rToyw3ngUs1s .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t856rToyw3ngUs1s .icon-shape p,#mermaid-svg-t856rToyw3ngUs1s .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-t856rToyw3ngUs1s .icon-shape .label rect,#mermaid-svg-t856rToyw3ngUs1s .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t856rToyw3ngUs1s .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-t856rToyw3ngUs1s .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-t856rToyw3ngUs1s :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 外部依赖
Kubernetes Node
nvidia-device-plugin (DaemonSet)
gRPC (Unix Socket)
Kubelet
main.go
入口 + 生命周期管理
plugin-manager.go
插件工厂
root.go
路径解析工具
server.go
gRPC 服务 + Device Plugin API
factory.go
插件构造器
api.go
Interface 定义
mps.go
MPS 选项
options.go
Option 模式配置
go-nvml
NVIDIA 驱动接口
container-device-interface
CDI 规范生成
internal/rm
资源管理器
internal/watch
文件/信号监控
api/config/v1
配置规范
核心插件模块处于 kubelet ↔ GPU 硬件 的中间层：向上通过 gRPC 与 kubelet 通信，向下通过 NVML/设备文件与 GPU 硬件交互。
二、模块整体结构
2.1 接口定义与类型体系
#mermaid-svg-LDyOHGGWYdH93N3z{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-LDyOHGGWYdH93N3z .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-LDyOHGGWYdH93N3z .error-icon{fill:#552222;}#mermaid-svg-LDyOHGGWYdH93N3z .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-LDyOHGGWYdH93N3z .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-LDyOHGGWYdH93N3z .marker{fill:#333333;stroke:#333333;}#mermaid-svg-LDyOHGGWYdH93N3z .marker.cross{stroke:#333333;}#mermaid-svg-LDyOHGGWYdH93N3z svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-LDyOHGGWYdH93N3z p{margin:0;}#mermaid-svg-LDyOHGGWYdH93N3z g.classGroup text{fill:#9370DB;stroke:none;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:10px;}#mermaid-svg-LDyOHGGWYdH93N3z g.classGroup text .title{font-weight:bolder;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster-label text{fill:#333;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster-label span{color:#333;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster-label span p{background-color:transparent;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster text{fill:#333;}#mermaid-svg-LDyOHGGWYdH93N3z .cluster span{color:#333;}#mermaid-svg-LDyOHGGWYdH93N3z .nodeLabel,#mermaid-svg-LDyOHGGWYdH93N3z .edgeLabel{color:#131300;}#mermaid-svg-LDyOHGGWYdH93N3z .edgeLabel .label rect{fill:#ECECFF;}#mermaid-svg-LDyOHGGWYdH93N3z .label text{fill:#131300;}#mermaid-svg-LDyOHGGWYdH93N3z .labelBkg{background:#ECECFF;}#mermaid-svg-LDyOHGGWYdH93N3z .edgeLabel .label span{background:#ECECFF;}#mermaid-svg-LDyOHGGWYdH93N3z .classTitle{font-weight:bolder;}#mermaid-svg-LDyOHGGWYdH93N3z .node rect,#mermaid-svg-LDyOHGGWYdH93N3z .node circle,#mermaid-svg-LDyOHGGWYdH93N3z .node ellipse,#mermaid-svg-LDyOHGGWYdH93N3z .node polygon,#mermaid-svg-LDyOHGGWYdH93N3z .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-LDyOHGGWYdH93N3z .divider{stroke:#9370DB;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z g.clickable{cursor:pointer;}#mermaid-svg-LDyOHGGWYdH93N3z g.classGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-LDyOHGGWYdH93N3z g.classGroup line{stroke:#9370DB;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-LDyOHGGWYdH93N3z .classLabel .label{fill:#9370DB;font-size:10px;}#mermaid-svg-LDyOHGGWYdH93N3z .relation{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-LDyOHGGWYdH93N3z .dashed-line{stroke-dasharray:3;}#mermaid-svg-LDyOHGGWYdH93N3z .dotted-line{stroke-dasharray:1 2;}#mermaid-svg-LDyOHGGWYdH93N3z #compositionStart,#mermaid-svg-LDyOHGGWYdH93N3z .composition{fill:#333333!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #compositionEnd,#mermaid-svg-LDyOHGGWYdH93N3z .composition{fill:#333333!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #dependencyStart,#mermaid-svg-LDyOHGGWYdH93N3z .dependency{fill:#333333!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #dependencyStart,#mermaid-svg-LDyOHGGWYdH93N3z .dependency{fill:#333333!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #extensionStart,#mermaid-svg-LDyOHGGWYdH93N3z .extension{fill:transparent!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #extensionEnd,#mermaid-svg-LDyOHGGWYdH93N3z .extension{fill:transparent!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #aggregationStart,#mermaid-svg-LDyOHGGWYdH93N3z .aggregation{fill:transparent!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #aggregationEnd,#mermaid-svg-LDyOHGGWYdH93N3z .aggregation{fill:transparent!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #lollipopStart,#mermaid-svg-LDyOHGGWYdH93N3z .lollipop{fill:#ECECFF!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z #lollipopEnd,#mermaid-svg-LDyOHGGWYdH93N3z .lollipop{fill:#ECECFF!important;stroke:#333333!important;stroke-width:1;}#mermaid-svg-LDyOHGGWYdH93N3z .edgeTerminals{font-size:11px;line-height:initial;}#mermaid-svg-LDyOHGGWYdH93N3z .classTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-LDyOHGGWYdH93N3z .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-LDyOHGGWYdH93N3z .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-LDyOHGGWYdH93N3z :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} implements
contains
creates
creates
<<interface>>
Interface
+Devices() : rm.Devices
+Start(socket string) : error
+Stop() : error
nvidiaDevicePlugin
-ctx context.Context
-rm rm.ResourceManager
-config *spec.Config
-deviceListStrategies spec.DeviceListStrategies
-cdiHandler cdi.Interface
-cdiAnnotationPrefix string
-socket string
-server *grpc.Server
-health chan *rm.Device
-stop chan interface
-imexChannels imex.Channels
-mps mpsOptions
+Initialize()
+Cleanup()
+Serve() : error
+Register(kubeletSocket) : error
+ListAndWatch(e, s) : error
+GetPreferredAllocation(ctx, r)(*PreferredAllocationResponse, error)
+Allocate(ctx, reqs)(*AllocateResponse, error)
+PreStartContainer(ctx, req)(*PreStartContainerResponse, error)
-getAllocateResponse(ids)(*ContainerAllocateResponse, error)
-updateResponseForCDI(resp, id, ids) : error
-updateResponseForMPS(resp)
-updateResponseForDeviceListEnvVar(resp, ids)
-updateResponseForDeviceMounts(resp, ids)
-apiDeviceSpecs(devRoot, ids) : \[\]*DeviceSpec
-dial(socket, timeout)(*ClientConn, error)
options
-infolib info.Interface
-nvmllib nvml.Interface
-devicelib device.Interface
-failOnInitError bool
-cdiHandler cdi.Interface
-config *spec.Config
-deviceListStrategies spec.DeviceListStrategies
-deviceDiscoveryStrategy string
-imexChannels imex.Channels
+getResourceManagers()(\[\]ResourceManager, error)
+devicePluginForResource(ctx, rm)(Interface, error)
+getMPSOptions(rm)(mpsOptions, error)
mpsOptions
+enabled bool
+resourceName spec.ResourceName
+daemon *mps.Daemon
+hostRoot mps.Root
+waitForDaemon() : error
+updateReponse(resp)
<<type alias: string>>
root
+join(parts) : string
+getDevRoot() : string
+isDevRoot() : bool
+tryResolveLibrary(name) : string
2.2 核心方法清单
| 文件 | 方法 | 作用 |
|---|---|---|
main.go |
main() |
程序入口，定义 CLI 参数，启动应用 |
main.go |
start() |
主事件循环，管理插件生命周期 |
main.go |
startPlugins() |
加载配置、创建并启动所有插件 |
main.go |
stopPlugins() |
停止所有插件 |
main.go |
validateFlags() |
校验 CLI/配置参数合法性 |
main.go |
loadConfig() |
从 CLI 上下文加载配置 |
plugin-manager.go |
GetPlugins() |
工厂方法：创建 CDI Handler + Plugin 集合 |
plugin-manager.go |
resolveStrategy() |
将 "auto" 策略解析为具体的 nvml/tegra |
root.go |
join() |
路径拼接 |
root.go |
getDevRoot() |
获取设备根路径 |
root.go |
tryResolveLibrary() |
在 driver root 下解析动态库路径 |
server.go |
Start() |
启动 gRPC 服务 + 注册到 kubelet + 启动健康检查 |
server.go |
Stop() |
停止 gRPC 服务 + 清理 socket 文件 |
server.go |
Serve() |
启动 gRPC server（含崩溃重启逻辑） |
server.go |
Register() |
向 kubelet 注册设备插件 |
server.go |
ListAndWatch() |
持续推送设备列表和健康状态 |
server.go |
Allocate() |
处理设备分配请求 |
server.go |
GetPreferredAllocation() |
返回推荐分配方案 |
factory.go |
New() |
插件工厂入口：创建所有插件实例 |
factory.go |
getResourceManagers() |
根据策略创建资源管理器 |
mps.go |
getMPSOptions() |
构造 MPS 选项 |
mps.go |
waitForDaemon() |
等待 MPS 守护进程就绪 |
mps.go |
updateReponse() |
向分配响应注入 MPS 挂载和环境变量 |
2.3 内部调用关系
#mermaid-svg-JG3bEus4zgm8KNNa{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-JG3bEus4zgm8KNNa .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-JG3bEus4zgm8KNNa .error-icon{fill:#552222;}#mermaid-svg-JG3bEus4zgm8KNNa .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-JG3bEus4zgm8KNNa .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-JG3bEus4zgm8KNNa .marker{fill:#333333;stroke:#333333;}#mermaid-svg-JG3bEus4zgm8KNNa .marker.cross{stroke:#333333;}#mermaid-svg-JG3bEus4zgm8KNNa svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-JG3bEus4zgm8KNNa p{margin:0;}#mermaid-svg-JG3bEus4zgm8KNNa .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-JG3bEus4zgm8KNNa .cluster-label text{fill:#333;}#mermaid-svg-JG3bEus4zgm8KNNa .cluster-label span{color:#333;}#mermaid-svg-JG3bEus4zgm8KNNa .cluster-label span p{background-color:transparent;}#mermaid-svg-JG3bEus4zgm8KNNa .label text,#mermaid-svg-JG3bEus4zgm8KNNa span{fill:#333;color:#333;}#mermaid-svg-JG3bEus4zgm8KNNa .node rect,#mermaid-svg-JG3bEus4zgm8KNNa .node circle,#mermaid-svg-JG3bEus4zgm8KNNa .node ellipse,#mermaid-svg-JG3bEus4zgm8KNNa .node polygon,#mermaid-svg-JG3bEus4zgm8KNNa .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-JG3bEus4zgm8KNNa .rough-node .label text,#mermaid-svg-JG3bEus4zgm8KNNa .node .label text,#mermaid-svg-JG3bEus4zgm8KNNa .image-shape .label,#mermaid-svg-JG3bEus4zgm8KNNa .icon-shape .label{text-anchor:middle;}#mermaid-svg-JG3bEus4zgm8KNNa .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-JG3bEus4zgm8KNNa .rough-node .label,#mermaid-svg-JG3bEus4zgm8KNNa .node .label,#mermaid-svg-JG3bEus4zgm8KNNa .image-shape .label,#mermaid-svg-JG3bEus4zgm8KNNa .icon-shape .label{text-align:center;}#mermaid-svg-JG3bEus4zgm8KNNa .node.clickable{cursor:pointer;}#mermaid-svg-JG3bEus4zgm8KNNa .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-JG3bEus4zgm8KNNa .arrowheadPath{fill:#333333;}#mermaid-svg-JG3bEus4zgm8KNNa .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-JG3bEus4zgm8KNNa .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-JG3bEus4zgm8KNNa .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JG3bEus4zgm8KNNa .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-JG3bEus4zgm8KNNa .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JG3bEus4zgm8KNNa .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-JG3bEus4zgm8KNNa .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-JG3bEus4zgm8KNNa .cluster text{fill:#333;}#mermaid-svg-JG3bEus4zgm8KNNa .cluster span{color:#333;}#mermaid-svg-JG3bEus4zgm8KNNa div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-JG3bEus4zgm8KNNa .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-JG3bEus4zgm8KNNa rect.text{fill:none;stroke-width:0;}#mermaid-svg-JG3bEus4zgm8KNNa .icon-shape,#mermaid-svg-JG3bEus4zgm8KNNa .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JG3bEus4zgm8KNNa .icon-shape p,#mermaid-svg-JG3bEus4zgm8KNNa .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-JG3bEus4zgm8KNNa .icon-shape .label rect,#mermaid-svg-JG3bEus4zgm8KNNa .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JG3bEus4zgm8KNNa .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-JG3bEus4zgm8KNNa .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-JG3bEus4zgm8KNNa :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} nvml
tegra
main()
start()
watch.Files()
watch.Signals()
startPlugins()
loadConfig()
validateFlags()
rm.AddDefaultResourcesToConfig()
GetPlugins()
resolveStrategy()
cdi.New()
plugin.New()
getResourceManagers()
rm.NewNVMLResourceManagers()
rm.NewTegraResourceManagers()
devicePluginForResource()
getMPSOptions()
cdiHandler.CreateSpecFile()
p.Start()
mps.waitForDaemon()
Serve()
Register()
rm.CheckHealth()
stopPlugins()
p.Stop()
2.4 数据流入流出
#mermaid-svg-7RN5oT4ORYp1A4HD{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-7RN5oT4ORYp1A4HD .error-icon{fill:#552222;}#mermaid-svg-7RN5oT4ORYp1A4HD .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-7RN5oT4ORYp1A4HD .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-7RN5oT4ORYp1A4HD .marker{fill:#333333;stroke:#333333;}#mermaid-svg-7RN5oT4ORYp1A4HD .marker.cross{stroke:#333333;}#mermaid-svg-7RN5oT4ORYp1A4HD svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-7RN5oT4ORYp1A4HD p{margin:0;}#mermaid-svg-7RN5oT4ORYp1A4HD .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster-label text{fill:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster-label span{color:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster-label span p{background-color:transparent;}#mermaid-svg-7RN5oT4ORYp1A4HD .label text,#mermaid-svg-7RN5oT4ORYp1A4HD span{fill:#333;color:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD .node rect,#mermaid-svg-7RN5oT4ORYp1A4HD .node circle,#mermaid-svg-7RN5oT4ORYp1A4HD .node ellipse,#mermaid-svg-7RN5oT4ORYp1A4HD .node polygon,#mermaid-svg-7RN5oT4ORYp1A4HD .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-7RN5oT4ORYp1A4HD .rough-node .label text,#mermaid-svg-7RN5oT4ORYp1A4HD .node .label text,#mermaid-svg-7RN5oT4ORYp1A4HD .image-shape .label,#mermaid-svg-7RN5oT4ORYp1A4HD .icon-shape .label{text-anchor:middle;}#mermaid-svg-7RN5oT4ORYp1A4HD .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-7RN5oT4ORYp1A4HD .rough-node .label,#mermaid-svg-7RN5oT4ORYp1A4HD .node .label,#mermaid-svg-7RN5oT4ORYp1A4HD .image-shape .label,#mermaid-svg-7RN5oT4ORYp1A4HD .icon-shape .label{text-align:center;}#mermaid-svg-7RN5oT4ORYp1A4HD .node.clickable{cursor:pointer;}#mermaid-svg-7RN5oT4ORYp1A4HD .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-7RN5oT4ORYp1A4HD .arrowheadPath{fill:#333333;}#mermaid-svg-7RN5oT4ORYp1A4HD .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-7RN5oT4ORYp1A4HD .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-7RN5oT4ORYp1A4HD .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-7RN5oT4ORYp1A4HD .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-7RN5oT4ORYp1A4HD .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-7RN5oT4ORYp1A4HD .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster text{fill:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD .cluster span{color:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-7RN5oT4ORYp1A4HD .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-7RN5oT4ORYp1A4HD rect.text{fill:none;stroke-width:0;}#mermaid-svg-7RN5oT4ORYp1A4HD .icon-shape,#mermaid-svg-7RN5oT4ORYp1A4HD .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-7RN5oT4ORYp1A4HD .icon-shape p,#mermaid-svg-7RN5oT4ORYp1A4HD .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-7RN5oT4ORYp1A4HD .icon-shape .label rect,#mermaid-svg-7RN5oT4ORYp1A4HD .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-7RN5oT4ORYp1A4HD .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-7RN5oT4ORYp1A4HD .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-7RN5oT4ORYp1A4HD :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 输出
处理
输入
CLI 参数 / 环境变量
config.yaml
NVML 库 / GPU 硬件
kubelet gRPC 请求
loadConfig
validateFlags
ResourceManager
nvidiaDevicePlugin
CDI Handler
Unix Socket
(gRPC 服务)
RegisterResponse
ListAndWatchResponse
AllocateResponse
(Envs/Mounts/Devices/Annotations)
CDI Spec File
三、核心业务逻辑深度解析
3.1 程序入口：main()
go
func main() {
c := cli.NewApp()
o := &options{}
c.Name = "NVIDIA Device Plugin"
c.Usage = "NVIDIA device plugin for Kubernetes"
c.Version = info.GetVersionString()
c.Action = func(ctx *cli.Context) error {
return start(ctx, o)
}
// ... 定义 CLI flags ...
o.flags = c.Flags
err := c.Run(os.Args)
if err != nil {
klog.Error(err)
os.Exit(1)
}
}逐行解析：
c := cli.NewApp()--- 创建 urfave/cli 应用实例，用于处理命令行参数o := &options{}--- 创建 options 结构体，用于在main和start之间传递 CLI 特有参数（kubeletSocket、configFile、cdiFeatureFlags），这些参数没有 config file 等价物c.Action = func(ctx *cli.Context) error { return start(ctx, o) }--- 设置主 action 为start，传入 CLI 上下文和 optionsc.Flags = []cli.Flag{...}--- 定义所有 CLI flags，每个 flag 都支持环境变量，主要包括：--mig-strategy：MIG 策略（none/single/mixed）--fail-on-init-error：初始化失败时是否直接退出--driver-root/--dev-root：NVIDIA 驱动/设备根路径--pass-device-specs：是否传递 DeviceSpecs 到 kubelet--device-list-strategy：设备列表传递策略（envvar/volume-mounts/cdi-annotations/cdi-cri）--device-id-strategy：设备 ID 策略（uuid/index）--gdrcopy-enabled/--gds-enabled/--mofed-enabled：高级特性开关--config-file：配置文件路径--mps-root：MPS 根路径--device-discovery-strategy：设备发现策略（auto/nvml/tegra）--imex-channel-ids/--imex-required：IMEX 通道配置
c.Run(os.Args)--- 执行 CLI 应用，触发start
3.2 主事件循环：start()
#mermaid-svg-ciXTiHbJLLSkKTVA{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ciXTiHbJLLSkKTVA .error-icon{fill:#552222;}#mermaid-svg-ciXTiHbJLLSkKTVA .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ciXTiHbJLLSkKTVA .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ciXTiHbJLLSkKTVA .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA .marker.cross{stroke:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ciXTiHbJLLSkKTVA p{margin:0;}#mermaid-svg-ciXTiHbJLLSkKTVA defs #statediagram-barbEnd{fill:#333333;stroke:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#mermaid-svg-ciXTiHbJLLSkKTVA g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#mermaid-svg-ciXTiHbJLLSkKTVA g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#mermaid-svg-ciXTiHbJLLSkKTVA g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-ciXTiHbJLLSkKTVA g.stateGroup line{stroke:#333333;stroke-width:1;}#mermaid-svg-ciXTiHbJLLSkKTVA .transition{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-ciXTiHbJLLSkKTVA .stateGroup .composit{fill:white;border-bottom:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA .state-note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-ciXTiHbJLLSkKTVA .state-note text{fill:black;stroke:none;font-size:10px;}#mermaid-svg-ciXTiHbJLLSkKTVA .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-ciXTiHbJLLSkKTVA .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#mermaid-svg-ciXTiHbJLLSkKTVA .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ciXTiHbJLLSkKTVA .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-ciXTiHbJLLSkKTVA .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ciXTiHbJLLSkKTVA .edgeLabel .label text{fill:#333;}#mermaid-svg-ciXTiHbJLLSkKTVA .label div .edgeLabel{color:#333;}#mermaid-svg-ciXTiHbJLLSkKTVA .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#mermaid-svg-ciXTiHbJLLSkKTVA .node circle.state-start{fill:#333333;stroke:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA .node .fork-join{fill:#333333;stroke:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#mermaid-svg-ciXTiHbJLLSkKTVA .end-state-inner{fill:white;stroke-width:1.5;}#mermaid-svg-ciXTiHbJLLSkKTVA .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA #statediagram-barbEnd{fill:#333333;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ciXTiHbJLLSkKTVA .cluster-label,#mermaid-svg-ciXTiHbJLLSkKTVA .nodeLabel{color:#131300;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-cluster rect.outer{rx:5px;ry:5px;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-state .divider{stroke:#9370DB;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-state .title-state{rx:5px;ry:5px;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-cluster.statediagram-cluster .inner{fill:white;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-cluster .inner{rx:0;ry:0;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-state rect.basic{rx:5px;ry:5px;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#mermaid-svg-ciXTiHbJLLSkKTVA .note-edge{stroke-dasharray:5;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-note text{fill:black;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram-note .nodeLabel{color:black;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagram .edgeLabel{color:red;}#mermaid-svg-ciXTiHbJLLSkKTVA #dependencyStart,#mermaid-svg-ciXTiHbJLLSkKTVA #dependencyEnd{fill:#333333;stroke:#333333;stroke-width:1;}#mermaid-svg-ciXTiHbJLLSkKTVA .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-ciXTiHbJLLSkKTVA :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Watching
main()
kubelet socket 创建 / SIGHUP
fsnotify error (仅日志)
SIGINT/SIGTERM/SIGQUIT
restartTimeout (30s)
watcher + signals 就绪
Init
StartPlugins
EventLoop
Exiting
go
func start(c *cli.Context, o *options) error {
klog.InfoS(fmt.Sprintf("Starting %s", c.App.Name), "version", c.App.Version)
// 1. 创建文件系统 watcher，监控 kubelet socket 目录
kubeletSocketDir := filepath.Dir(o.kubeletSocket)
watcher, err := watch.Files(kubeletSocketDir)
// ...
defer watcher.Close()
// 2. 创建 OS 信号 watcher
sigs := watch.Signals(syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
var started bool
var restartTimeout <-chan time.Time
var plugins []plugin.Interface
restart:
// 3. 如果是重启，先停止旧插件
if started {
err := stopPlugins(plugins)
// ...
}
// 4. 启动插件
plugins, restartPlugins, err := startPlugins(c, o)
// ...
started = true
// 5. 如果部分插件启动失败，设置 30s 重试定时器
if restartPlugins {
restartTimeout = time.After(30 * time.Second)
}
// 6. 无限循环，监听多个事件源
for {
select {
case <-restartTimeout: // 重试超时 → 重启
goto restart
case event := <-watcher.Events: // kubelet socket 重建 → 重启
if o.kubeletSocket != "" && event.Name == o.kubeletSocket &&
event.Op&fsnotify.Create == fsnotify.Create {
goto restart
}
case err := <-watcher.Errors: // fsnotify 错误 → 仅记录日志
klog.Infof("inotify: %s", err)
case s := <-sigs: // OS 信号
switch s {
case syscall.SIGHUP: // SIGHUP → 重启
goto restart
default: // 其他信号 → 退出
goto exit
}
}
}
exit:
err = stopPlugins(plugins)
return err
}设计意图分析：
- kubelet 重启检测 ：kubelet 重启时会重新创建 socket 文件，通过 inotify 检测到
Create事件后自动重启插件，确保设备插件与 kubelet 始终保持连接 - SIGHUP 热重启：支持通过 SIGHUP 信号触发插件重启，用于配置变更后无需重启 Pod 即可生效
- 崩溃退避 ：
restartPlugins标志 + 30s 延迟，避免在 kubelet 未就绪时频繁重试 - 优雅退出：SIGINT/SIGTERM/SIGQUIT 触发优雅停止，先停止所有插件再退出
3.3 插件启动：startPlugins()
go
func startPlugins(c *cli.Context, o *options) ([]plugin.Interface, bool, error) {
// Step 1: 加载配置
config, err := loadConfig(c, o.flags)
// ...
spec.DisableResourceNamingInConfig(config)
// Step 2: 构造 NVML/device/info 库实例
driverRoot := root(*config.Flags.Plugin.ContainerDriverRoot)
nvmllib := nvml.New(
nvml.WithLibraryPath(driverRoot.tryResolveLibrary("libnvidia-ml.so.1")),
)
devicelib := device.New(nvmllib)
infolib := nvinfo.New(
nvinfo.WithRoot(string(driverRoot)),
nvinfo.WithNvmlLib(nvmllib),
nvinfo.WithDeviceLib(devicelib),
)
// Step 3: 校验 flags
err = validateFlags(infolib, config)
// ...
// Step 4: 添加默认资源到配置
err = rm.AddDefaultResourcesToConfig(infolib, nvmllib, devicelib, config)
// ...
// Step 5: 打印配置
configJSON, _ := json.MarshalIndent(config, "", " ")
klog.Infof("\nRunning with config:\n%v", string(configJSON))
// Step 6: 获取插件集合
plugins, err := GetPlugins(c.Context, infolib, nvmllib, devicelib, config, o)
// ...
// Step 7: 遍历插件，启动有设备的插件
started := 0
for _, p := range plugins {
if len(p.Devices()) == 0 {
continue // 无设备的插件跳过
}
if err := p.Start(o.kubeletSocket); err != nil {
return plugins, true, nil // 启动失败，返回需要重试
}
started++
}
if started == 0 {
klog.Info("No devices found. Waiting indefinitely.")
}
return plugins, false, nil
}关键设计点：
driverRoot.tryResolveLibrary("libnvidia-ml.so.1")--- 在 driver root 下搜索 NVML 库，支持 driver container 场景（如/run/nvidia/driver/usr/lib64/libnvidia-ml.so.1）spec.DisableResourceNamingInConfig(config)--- 禁用资源命名特性（可能与某些 K8s 版本不兼容）rm.AddDefaultResourcesToConfig()--- 根据硬件自动添加默认资源（如nvidia.com/gpu,nvidia.com/mig-1g.10gb等）- 只启动有设备的插件，无设备的插件被跳过（但仍然返回，以便统一管理生命周期）
3.4 配置校验：validateFlags()
go
func validateFlags(infolib nvinfo.Interface, config *spec.Config) error {
// 1. 解析并校验 device-list-strategy
deviceListStrategies, err := spec.NewDeviceListStrategies(*config.Flags.Plugin.DeviceListStrategy)
if err != nil {
return fmt.Errorf("invalid --device-list-strategy option: %v", err)
}
// 2. CDI 策略需要 NVML 支持
hasNvml, _ := infolib.HasNvml()
if deviceListStrategies.AnyCDIEnabled() && !hasNvml {
return fmt.Errorf("CDI --device-list-strategy options are only supported on NVML-based systems")
}
// 3. 校验 device-id-strategy（只能是 uuid 或 index）
if *config.Flags.Plugin.DeviceIDStrategy != spec.DeviceIDStrategyUUID &&
*config.Flags.Plugin.DeviceIDStrategy != spec.DeviceIDStrategyIndex {
return fmt.Errorf("invalid --device-id-strategy option: %v", ...)
}
// 4. MPS 共享策略校验
if config.Sharing.SharingStrategy() == spec.SharingStrategyMPS {
if *config.Flags.MigStrategy == spec.MigStrategyMixed {
return fmt.Errorf("using --mig-strategy=mixed is not supported with MPS")
}
if config.Flags.MpsRoot == nil || *config.Flags.MpsRoot == "" {
return fmt.Errorf("using MPS requires --mps-root to be specified")
}
}
// 5. 校验 device-discovery-strategy
switch *config.Flags.DeviceDiscoveryStrategy {
case "auto", "nvml", "tegra": // 合法值
default:
return fmt.Errorf("invalid --device-discovery-strategy option %v", ...)
}
// 6. 校验 mig-strategy
switch *config.Flags.MigStrategy {
case spec.MigStrategyNone, spec.MigStrategySingle, spec.MigStrategyMixed:
default:
return fmt.Errorf("unknown MIG strategy: %v", ...)
}
// 7. 校验 IMEX channel IDs
if err := spec.AssertChannelIDsValid(config.Imex.ChannelIDs); err != nil {
return fmt.Errorf("invalid IMEX channel IDs: %w", err)
}
return nil
}校验规则矩阵：
| 参数 | 合法值 | 约束条件 |
|---|---|---|
device-list-strategy |
envvar, volume-mounts, cdi-annotations, cdi-cri | CDI 策略需 NVML |
device-id-strategy |
uuid, index | --- |
mig-strategy |
none, single, mixed | mixed 不兼容 MPS |
device-discovery-strategy |
auto, nvml, tegra | --- |
mps-root |
非空字符串 | MPS 共享时必填 |
imex-channel-ids |
有效 channel ID | --- |
3.5 插件工厂：GetPlugins() (plugin-manager.go)
go
func GetPlugins(ctx context.Context, infolib info.Interface, nvmllib nvml.Interface,
devicelib device.Interface, config *spec.Config, o *options) ([]plugin.Interface, error) {
driverRoot := root(*config.Flags.Plugin.ContainerDriverRoot)
// 1. 解析设备列表策略
deviceListStrategies, err := spec.NewDeviceListStrategies(*config.Flags.Plugin.DeviceListStrategy)
// ...
// 2. 获取 IMEX 通道
imexChannels, err := imex.GetChannels(config, driverRoot.getDevRoot())
// ...
// 3. 解析设备发现策略（auto → nvml/tegra）
resolvedStrategy := resolveStrategy(*config.Flags.DeviceDiscoveryStrategy, infolib)
// 4. 构造 CDI Handler
cdiHandler, err := cdi.New(infolib, nvmllib, devicelib,
cdi.WithDeviceListStrategies(deviceListStrategies),
cdi.WithDriverRoot(string(driverRoot)),
cdi.WithDevRoot(driverRoot.getDevRoot()),
cdi.WithTargetDriverRoot(*config.Flags.NvidiaDriverRoot),
cdi.WithTargetDevRoot(*config.Flags.NvidiaDevRoot),
cdi.WithNvidiaCTKPath(*config.Flags.Plugin.NvidiaCTKPath),
cdi.WithDeviceIDStrategy(*config.Flags.Plugin.DeviceIDStrategy),
cdi.WithVendor("k8s.device-plugin.nvidia.com"),
cdi.WithGdrcopyEnabled(*config.Flags.GDRCopyEnabled),
cdi.WithGdsEnabled(*config.Flags.GDSEnabled),
cdi.WithMofedEnabled(*config.Flags.MOFEDEnabled),
cdi.WithImexChannels(imexChannels),
cdi.WithFeatureFlags(o.cdiFeatureFlags.Value()...),
cdi.WithDeviceDiscoveryStrategy(resolvedStrategy),
)
// ...
// 5. 构造插件集合
plugins, err := plugin.New(ctx, infolib, nvmllib, devicelib,
plugin.WithCDIHandler(cdiHandler),
plugin.WithConfig(config),
plugin.WithDeviceListStrategies(deviceListStrategies),
plugin.WithFailOnInitError(*config.Flags.FailOnInitError),
plugin.WithImexChannels(imexChannels),
plugin.WithDeviceDiscoveryStrategy(resolvedStrategy),
)
// ...
// 6. 生成 CDI Spec 文件
if err := cdiHandler.CreateSpecFile(); err != nil {
return nil, fmt.Errorf("unable to create cdi spec file: %v", err)
}
return plugins, nil
}resolveStrategy() 自动检测逻辑：
go
func resolveStrategy(strategy string, infolib info.Interface) string {
if strategy != "" && strategy != "auto" {
return strategy // 用户显式指定，直接返回
}
platform := infolib.ResolvePlatform()
switch platform {
case info.PlatformNVML, info.PlatformWSL:
return "nvml"
case info.PlatformTegra:
return "tegra"
}
return "nvml" // 默认回退到 nvml
}#mermaid-svg-ngpAmEbFLaQN9wad{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ngpAmEbFLaQN9wad .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ngpAmEbFLaQN9wad .error-icon{fill:#552222;}#mermaid-svg-ngpAmEbFLaQN9wad .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ngpAmEbFLaQN9wad .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ngpAmEbFLaQN9wad .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ngpAmEbFLaQN9wad .marker.cross{stroke:#333333;}#mermaid-svg-ngpAmEbFLaQN9wad svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ngpAmEbFLaQN9wad p{margin:0;}#mermaid-svg-ngpAmEbFLaQN9wad .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-ngpAmEbFLaQN9wad .cluster-label text{fill:#333;}#mermaid-svg-ngpAmEbFLaQN9wad .cluster-label span{color:#333;}#mermaid-svg-ngpAmEbFLaQN9wad .cluster-label span p{background-color:transparent;}#mermaid-svg-ngpAmEbFLaQN9wad .label text,#mermaid-svg-ngpAmEbFLaQN9wad span{fill:#333;color:#333;}#mermaid-svg-ngpAmEbFLaQN9wad .node rect,#mermaid-svg-ngpAmEbFLaQN9wad .node circle,#mermaid-svg-ngpAmEbFLaQN9wad .node ellipse,#mermaid-svg-ngpAmEbFLaQN9wad .node polygon,#mermaid-svg-ngpAmEbFLaQN9wad .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ngpAmEbFLaQN9wad .rough-node .label text,#mermaid-svg-ngpAmEbFLaQN9wad .node .label text,#mermaid-svg-ngpAmEbFLaQN9wad .image-shape .label,#mermaid-svg-ngpAmEbFLaQN9wad .icon-shape .label{text-anchor:middle;}#mermaid-svg-ngpAmEbFLaQN9wad .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-ngpAmEbFLaQN9wad .rough-node .label,#mermaid-svg-ngpAmEbFLaQN9wad .node .label,#mermaid-svg-ngpAmEbFLaQN9wad .image-shape .label,#mermaid-svg-ngpAmEbFLaQN9wad .icon-shape .label{text-align:center;}#mermaid-svg-ngpAmEbFLaQN9wad .node.clickable{cursor:pointer;}#mermaid-svg-ngpAmEbFLaQN9wad .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-ngpAmEbFLaQN9wad .arrowheadPath{fill:#333333;}#mermaid-svg-ngpAmEbFLaQN9wad .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-ngpAmEbFLaQN9wad .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-ngpAmEbFLaQN9wad .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ngpAmEbFLaQN9wad .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-ngpAmEbFLaQN9wad .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ngpAmEbFLaQN9wad .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-ngpAmEbFLaQN9wad .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-ngpAmEbFLaQN9wad .cluster text{fill:#333;}#mermaid-svg-ngpAmEbFLaQN9wad .cluster span{color:#333;}#mermaid-svg-ngpAmEbFLaQN9wad div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-ngpAmEbFLaQN9wad .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-ngpAmEbFLaQN9wad rect.text{fill:none;stroke-width:0;}#mermaid-svg-ngpAmEbFLaQN9wad .icon-shape,#mermaid-svg-ngpAmEbFLaQN9wad .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ngpAmEbFLaQN9wad .icon-shape p,#mermaid-svg-ngpAmEbFLaQN9wad .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-ngpAmEbFLaQN9wad .icon-shape .label rect,#mermaid-svg-ngpAmEbFLaQN9wad .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ngpAmEbFLaQN9wad .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-ngpAmEbFLaQN9wad .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-ngpAmEbFLaQN9wad :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} No
Yes
NVML / WSL
Tegra
Unknown
resolveStrategy()
strategy == 'auto'?
返回 strategy
infolib.ResolvePlatform()
Platform?
返回 'nvml'
返回 'tegra'
默认返回 'nvml'
3.6 插件构造器：plugin.New() (factory.go)
go
func New(ctx context.Context, infolib info.Interface, nvmllib nvml.Interface,
devicelib device.Interface, opts ...Option) ([]Interface, error) {
// 1. 初始化 options，应用所有 Option 函数
o := &options{
infolib: infolib,
nvmllib: nvmllib,
devicelib: devicelib,
}
for _, opt := range opts {
opt(o)
}
// 2. 校验 config
if o.config == nil {
klog.Warning("no config provided, returning a null manager")
return nil, nil
}
// 3. 默认 CDI Handler
if o.cdiHandler == nil {
o.cdiHandler = cdi.NewNullHandler()
}
// 4. 校验 discovery strategy
if o.deviceDiscoveryStrategy == "" {
return nil, fmt.Errorf("device discovery strategy not set")
}
// 5. 获取资源管理器集合
resourceManagers, err := o.getResourceManagers()
// ...
// 6. 为每个资源管理器创建一个插件实例
var plugins []Interface
for _, resourceManager := range resourceManagers {
plugin, err := o.devicePluginForResource(ctx, resourceManager)
// ...
plugins = append(plugins, plugin)
}
return plugins, nil
}getResourceManagers() --- 设备发现核心：
go
func (o *options) getResourceManagers() ([]rm.ResourceManager, error) {
strategy := o.deviceDiscoveryStrategy
switch strategy {
case "nvml":
// 初始化 NVML 库
ret := o.nvmllib.Init()
if ret != nvml.SUCCESS {
// 打印详细的错误帮助信息
klog.Errorf("Failed to initialize NVML: %v.", ret)
klog.Errorf("If this is a GPU node, did you set the docker default runtime to `nvidia`?")
// ...
if o.failOnInitError {
return nil, fmt.Errorf("nvml init failed: %v", ret)
}
klog.Warningf("nvml init failed: %v", ret)
return nil, nil // 不崩溃，返回空（等待重试）
}
defer func() {
_ = o.nvmllib.Shutdown()
}()
return rm.NewNVMLResourceManagers(o.infolib, o.nvmllib, o.devicelib, o.config)
case "tegra":
return rm.NewTegraResourceManagers(o.config)
default:
// 不支持的策略
if o.failOnInitError {
return nil, fmt.Errorf("invalid device discovery strategy")
}
return nil, nil
}
}failOnInitError 的双重行为：
true（默认）：NVML 初始化失败 → 返回错误 → 30s 后重试false：返回 nil → 无设备 → 等待（适用于非 GPU 节点）
3.7 Option 模式配置 (options.go)
go
type Option func(*options)
func WithCDIHandler(handler cdi.Interface) Option {
return func(m *options) { m.cdiHandler = handler }
}
func WithConfig(config *spec.Config) Option {
return func(m *options) { m.config = config }
}
// ... 其他 With* 函数采用经典的 Functional Options Pattern ，每个 With* 函数返回一个闭包，修改 options 结构体的对应字段。这使得 plugin.New() 的 API 可扩展且向后兼容。
#mermaid-svg-vq1EwXxxVLPUDBMd{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-vq1EwXxxVLPUDBMd .error-icon{fill:#552222;}#mermaid-svg-vq1EwXxxVLPUDBMd .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-vq1EwXxxVLPUDBMd .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-vq1EwXxxVLPUDBMd .marker{fill:#333333;stroke:#333333;}#mermaid-svg-vq1EwXxxVLPUDBMd .marker.cross{stroke:#333333;}#mermaid-svg-vq1EwXxxVLPUDBMd svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-vq1EwXxxVLPUDBMd p{margin:0;}#mermaid-svg-vq1EwXxxVLPUDBMd .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster-label text{fill:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster-label span{color:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster-label span p{background-color:transparent;}#mermaid-svg-vq1EwXxxVLPUDBMd .label text,#mermaid-svg-vq1EwXxxVLPUDBMd span{fill:#333;color:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd .node rect,#mermaid-svg-vq1EwXxxVLPUDBMd .node circle,#mermaid-svg-vq1EwXxxVLPUDBMd .node ellipse,#mermaid-svg-vq1EwXxxVLPUDBMd .node polygon,#mermaid-svg-vq1EwXxxVLPUDBMd .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-vq1EwXxxVLPUDBMd .rough-node .label text,#mermaid-svg-vq1EwXxxVLPUDBMd .node .label text,#mermaid-svg-vq1EwXxxVLPUDBMd .image-shape .label,#mermaid-svg-vq1EwXxxVLPUDBMd .icon-shape .label{text-anchor:middle;}#mermaid-svg-vq1EwXxxVLPUDBMd .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-vq1EwXxxVLPUDBMd .rough-node .label,#mermaid-svg-vq1EwXxxVLPUDBMd .node .label,#mermaid-svg-vq1EwXxxVLPUDBMd .image-shape .label,#mermaid-svg-vq1EwXxxVLPUDBMd .icon-shape .label{text-align:center;}#mermaid-svg-vq1EwXxxVLPUDBMd .node.clickable{cursor:pointer;}#mermaid-svg-vq1EwXxxVLPUDBMd .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-vq1EwXxxVLPUDBMd .arrowheadPath{fill:#333333;}#mermaid-svg-vq1EwXxxVLPUDBMd .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-vq1EwXxxVLPUDBMd .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-vq1EwXxxVLPUDBMd .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-vq1EwXxxVLPUDBMd .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-vq1EwXxxVLPUDBMd .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-vq1EwXxxVLPUDBMd .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster text{fill:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd .cluster span{color:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-vq1EwXxxVLPUDBMd .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-vq1EwXxxVLPUDBMd rect.text{fill:none;stroke-width:0;}#mermaid-svg-vq1EwXxxVLPUDBMd .icon-shape,#mermaid-svg-vq1EwXxxVLPUDBMd .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-vq1EwXxxVLPUDBMd .icon-shape p,#mermaid-svg-vq1EwXxxVLPUDBMd .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-vq1EwXxxVLPUDBMd .icon-shape .label rect,#mermaid-svg-vq1EwXxxVLPUDBMd .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-vq1EwXxxVLPUDBMd .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-vq1EwXxxVLPUDBMd .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-vq1EwXxxVLPUDBMd :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} plugin.New()
Option 函数们
WithCDIHandler()
WithConfig()
WithDeviceListStrategies()
WithFailOnInitError()
WithImexChannels()
WithDeviceDiscoveryStrategy()
WithNVML()
WithInfoLib()
o := &options{infolib, nvmllib, devicelib}
for _, opt := range opts { opt(o) }
o 完全配置
3.8 插件实例创建：devicePluginForResource()
go
func (o *options) devicePluginForResource(ctx context.Context,
resourceManager rm.ResourceManager) (Interface, error) {
// 1. 获取 MPS 选项
mpsOptions, err := o.getMPSOptions(resourceManager)
// ...
// 2. 构造 nvidiaDevicePlugin 实例
plugin := nvidiaDevicePlugin{
ctx: ctx,
rm: resourceManager,
config: o.config,
deviceListStrategies: o.deviceListStrategies,
cdiHandler: o.cdiHandler,
cdiAnnotationPrefix: *o.config.Flags.Plugin.CDIAnnotationPrefix,
imexChannels: o.imexChannels,
mps: mpsOptions,
socket: getPluginSocketPath(resourceManager.Resource()),
// 以下字段在每次 Start 时重新初始化
server: nil,
health: nil,
stop: nil,
}
return &plugin, nil
}
func getPluginSocketPath(resource spec.ResourceName) string {
_, name := resource.Split()
pluginName := "nvidia-" + name
return filepath.Join(pluginapi.DevicePluginPath, pluginName) + ".sock"
}Socket 路径规则： /var/lib/kubelet/device-plugins/nvidia-<resource-name>.sock
nvidia.com/gpu→/var/lib/kubelet/device-plugins/nvidia-gpu.socknvidia.com/mig-1g.10gb→/var/lib/kubelet/device-plugins/nvidia-mig-1g.10gb.sock
每个资源对应一个独立的插件实例和 gRPC server，kubelet 通过不同的 socket 与各资源类型的插件通信。
3.9 插件启动：Start()
Health Checker gRPC Server nvidiaDevicePlugin Kubelet Health Checker gRPC Server nvidiaDevicePlugin Kubelet #mermaid-svg-RoL1WQE17O8ZLOaq{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-RoL1WQE17O8ZLOaq .error-icon{fill:#552222;}#mermaid-svg-RoL1WQE17O8ZLOaq .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-RoL1WQE17O8ZLOaq .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-RoL1WQE17O8ZLOaq .marker{fill:#333333;stroke:#333333;}#mermaid-svg-RoL1WQE17O8ZLOaq .marker.cross{stroke:#333333;}#mermaid-svg-RoL1WQE17O8ZLOaq svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-RoL1WQE17O8ZLOaq p{margin:0;}#mermaid-svg-RoL1WQE17O8ZLOaq .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RoL1WQE17O8ZLOaq text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-RoL1WQE17O8ZLOaq .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-RoL1WQE17O8ZLOaq .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-RoL1WQE17O8ZLOaq #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-RoL1WQE17O8ZLOaq .sequenceNumber{fill:white;}#mermaid-svg-RoL1WQE17O8ZLOaq #sequencenumber{fill:#333;}#mermaid-svg-RoL1WQE17O8ZLOaq #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-RoL1WQE17O8ZLOaq .messageText{fill:#333;stroke:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RoL1WQE17O8ZLOaq .labelText,#mermaid-svg-RoL1WQE17O8ZLOaq .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .loopText,#mermaid-svg-RoL1WQE17O8ZLOaq .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-RoL1WQE17O8ZLOaq .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-RoL1WQE17O8ZLOaq .noteText,#mermaid-svg-RoL1WQE17O8ZLOaq .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-RoL1WQE17O8ZLOaq .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RoL1WQE17O8ZLOaq .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RoL1WQE17O8ZLOaq .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RoL1WQE17O8ZLOaq .actorPopupMenu{position:absolute;}#mermaid-svg-RoL1WQE17O8ZLOaq .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-RoL1WQE17O8ZLOaq .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RoL1WQE17O8ZLOaq .actor-man circle,#mermaid-svg-RoL1WQE17O8ZLOaq line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-RoL1WQE17O8ZLOaq :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 创建 grpc.Server创建 health channel创建 stop channel 监听 Unix socket注册 DevicePluginServer启动崩溃重启 goroutine 发送 RegisterRequest(Version, Endpoint, ResourceName, Options) 持续监控 GPU 健康通过 health channel 推送状态 initialize()mps.waitForDaemon()Serve()server 就绪 (dial 验证)Register()注册成功go rm.CheckHealth()
go
func (plugin *nvidiaDevicePlugin) Start(kubeletSocket string) error {
plugin.initialize() // ① 初始化 server/health/stop
if err := plugin.mps.waitForDaemon(); err != nil { // ② 等待 MPS daemon
return fmt.Errorf("error waiting for MPS daemon: %w", err)
}
err := plugin.Serve() // ③ 启动 gRPC 服务
if err != nil {
plugin.cleanup()
return err
}
err = plugin.Register(kubeletSocket) // ④ 注册到 kubelet
if err != nil {
return errors.Join(err, plugin.Stop())
}
go func() { // ⑤ 启动健康检查 goroutine
err := plugin.rm.CheckHealth(plugin.stop, plugin.health)
if err != nil {
klog.Errorf("Failed to start health check: %v; continuing with health checks disabled", err)
}
}()
return nil
}initialize() / cleanup() 配对：
go
func (plugin *nvidiaDevicePlugin) initialize() {
plugin.server = grpc.NewServer([]grpc.ServerOption{}...)
plugin.health = make(chan *rm.Device)
plugin.stop = make(chan interface{})
}
func (plugin *nvidiaDevicePlugin) cleanup() {
close(plugin.stop)
plugin.server = nil
plugin.health = nil
plugin.stop = nil
}initialize创建三个运行时组件：gRPC server、health channel（接收健康状态变更）、stop channel（广播停止信号）cleanup关闭 stop channel 并置空所有运行时字段，使插件回到可重新 Start 的状态
3.10 gRPC 服务启动：Serve()
go
func (plugin *nvidiaDevicePlugin) Serve() error {
os.Remove(plugin.socket) // 清理残留 socket
sock, err := net.Listen("unix", plugin.socket) // 监听 Unix socket
if err != nil {
return err
}
pluginapi.RegisterDevicePluginServer(plugin.server, plugin) // 注册 gRPC 服务
go func() { // 启动 gRPC server（含崩溃重启）
lastCrashTime := time.Now()
restartCount := 0
for {
// 防止频繁崩溃：5 次崩溃且每次存活 < 1 小时 → 退出
if restartCount > 5 {
klog.Fatalf("GRPC server for '%s' has repeatedly crashed recently. Quitting",
plugin.rm.Resource())
}
klog.Infof("Starting GRPC server for '%s'", plugin.rm.Resource())
err := plugin.server.Serve(sock)
if err == nil {
break // 正常停止
}
klog.Infof("GRPC server for '%s' crashed with error: %v", plugin.rm.Resource(), err)
timeSinceLastCrash := time.Since(lastCrashTime).Seconds()
lastCrashTime = time.Now()
if timeSinceLastCrash > 3600 {
restartCount = 0 // 超过 1 小时，重置计数
} else {
restartCount++
}
}
}()
// 阻塞等待 server 启动完成
conn, err := plugin.dial(plugin.socket, 5*time.Second)
if err != nil {
return err
}
conn.Close()
return nil
}崩溃重启策略：
#mermaid-svg-TJXe3FzDeyxNkO0H{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-TJXe3FzDeyxNkO0H .error-icon{fill:#552222;}#mermaid-svg-TJXe3FzDeyxNkO0H .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-TJXe3FzDeyxNkO0H .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-TJXe3FzDeyxNkO0H .marker{fill:#333333;stroke:#333333;}#mermaid-svg-TJXe3FzDeyxNkO0H .marker.cross{stroke:#333333;}#mermaid-svg-TJXe3FzDeyxNkO0H svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-TJXe3FzDeyxNkO0H p{margin:0;}#mermaid-svg-TJXe3FzDeyxNkO0H .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster-label text{fill:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster-label span{color:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster-label span p{background-color:transparent;}#mermaid-svg-TJXe3FzDeyxNkO0H .label text,#mermaid-svg-TJXe3FzDeyxNkO0H span{fill:#333;color:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H .node rect,#mermaid-svg-TJXe3FzDeyxNkO0H .node circle,#mermaid-svg-TJXe3FzDeyxNkO0H .node ellipse,#mermaid-svg-TJXe3FzDeyxNkO0H .node polygon,#mermaid-svg-TJXe3FzDeyxNkO0H .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-TJXe3FzDeyxNkO0H .rough-node .label text,#mermaid-svg-TJXe3FzDeyxNkO0H .node .label text,#mermaid-svg-TJXe3FzDeyxNkO0H .image-shape .label,#mermaid-svg-TJXe3FzDeyxNkO0H .icon-shape .label{text-anchor:middle;}#mermaid-svg-TJXe3FzDeyxNkO0H .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-TJXe3FzDeyxNkO0H .rough-node .label,#mermaid-svg-TJXe3FzDeyxNkO0H .node .label,#mermaid-svg-TJXe3FzDeyxNkO0H .image-shape .label,#mermaid-svg-TJXe3FzDeyxNkO0H .icon-shape .label{text-align:center;}#mermaid-svg-TJXe3FzDeyxNkO0H .node.clickable{cursor:pointer;}#mermaid-svg-TJXe3FzDeyxNkO0H .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-TJXe3FzDeyxNkO0H .arrowheadPath{fill:#333333;}#mermaid-svg-TJXe3FzDeyxNkO0H .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-TJXe3FzDeyxNkO0H .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-TJXe3FzDeyxNkO0H .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-TJXe3FzDeyxNkO0H .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-TJXe3FzDeyxNkO0H .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-TJXe3FzDeyxNkO0H .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster text{fill:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H .cluster span{color:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-TJXe3FzDeyxNkO0H .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-TJXe3FzDeyxNkO0H rect.text{fill:none;stroke-width:0;}#mermaid-svg-TJXe3FzDeyxNkO0H .icon-shape,#mermaid-svg-TJXe3FzDeyxNkO0H .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-TJXe3FzDeyxNkO0H .icon-shape p,#mermaid-svg-TJXe3FzDeyxNkO0H .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-TJXe3FzDeyxNkO0H .icon-shape .label rect,#mermaid-svg-TJXe3FzDeyxNkO0H .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-TJXe3FzDeyxNkO0H .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-TJXe3FzDeyxNkO0H .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-TJXe3FzDeyxNkO0H :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
Yes
No
Yes
No
启动 goroutine
restartCount > 5?
klog.Fatalf 退出
plugin.server.Serve(sock)
err == nil?
正常退出 goroutine
计算距上次崩溃时间
> 3600s?
restartCount = 0
restartCount++
设计意图： 防止 gRPC server 反复崩溃导致 CPU 飙升。通过 lastCrashTime + restartCount 组合实现退避------存活超 1 小时则重置计数；频繁崩溃超 5 次则 Fatal 退出（依赖 K8s 重启 Pod）。
3.11 注册到 Kubelet：Register()
go
func (plugin *nvidiaDevicePlugin) Register(kubeletSocket string) error {
if kubeletSocket == "" {
klog.Info("Skipping registration with Kubelet")
return nil // 空 socket → 跳过注册（用于测试/调试场景）
}
conn, err := plugin.dial(kubeletSocket, 5*time.Second) // 连接 kubelet
// ...
defer conn.Close()
client := pluginapi.NewRegistrationClient(conn)
reqt := &pluginapi.RegisterRequest{
Version: pluginapi.Version, // Device Plugin API 版本
Endpoint: path.Base(plugin.socket), // 插件 socket 文件名
ResourceName: string(plugin.rm.Resource()), // 如 "nvidia.com/gpu"
Options: &pluginapi.DevicePluginOptions{
GetPreferredAllocationAvailable: true, // 声明支持 PreferredAllocation
},
}
_, err = client.Register(plugin.ctx, reqt)
return err
}注册请求字段说明：
| 字段 | 值 | 说明 |
|---|---|---|
Version |
v1beta1 |
K8s Device Plugin API 版本 |
Endpoint |
nvidia-gpu.sock |
kubelet 通过此路径连接插件 |
ResourceName |
nvidia.com/gpu 等 |
注册为 K8s Extended Resource |
GetPreferredAllocationAvailable |
true |
告知 kubelet 本插件实现了 GetPreferredAllocation |
3.12 设备列表与健康监控：ListAndWatch()
go
func (plugin *nvidiaDevicePlugin) ListAndWatch(e *pluginapi.Empty,
s pluginapi.DevicePlugin_ListAndWatchServer) error {
// 1. 首次推送完整设备列表
if err := s.Send(&pluginapi.ListAndWatchResponse{Devices: plugin.apiDevices()}); err != nil {
return err
}
// 2. 持续监听健康状态变更
for {
select {
case <-plugin.stop: // 插件停止 → 退出
return nil
case d := <-plugin.health: // 收到健康状态变更
// FIXME: 不可恢复 --- Unhealthy 是终态
d.Health = pluginapi.Unhealthy
klog.Infof("'%s' device marked unhealthy: %s", plugin.rm.Resource(), d.ID)
if err := s.Send(&pluginapi.ListAndWatchResponse{Devices: plugin.apiDevices()}); err != nil {
return nil
}
}
}
}ResourceManager Health Checker (goroutine) ListAndWatch() Kubelet ResourceManager Health Checker (goroutine) ListAndWatch() Kubelet #mermaid-svg-gbLJ5wJnQgy2mtQ3{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .error-icon{fill:#552222;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .marker.cross{stroke:#333333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 p{margin:0;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .sequenceNumber{fill:white;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 #sequencenumber{fill:#333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .messageText{fill:#333;stroke:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .labelText,#mermaid-svg-gbLJ5wJnQgy2mtQ3 .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .loopText,#mermaid-svg-gbLJ5wJnQgy2mtQ3 .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .noteText,#mermaid-svg-gbLJ5wJnQgy2mtQ3 .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actorPopupMenu{position:absolute;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 .actor-man circle,#mermaid-svg-gbLJ5wJnQgy2mtQ3 line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-gbLJ5wJnQgy2mtQ3 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} loop持续监控 stop channel 关闭时退出 s.Send(完整设备列表)进入 select 循环检测到设备异常health channel <- Device{unhealthy}d.Health = Unhealthys.Send(更新后设备列表)
关键设计点：
- 首次调用必须发送完整设备列表，kubelet 据此建立设备清单
- 健康状态变更是增量推送------只在 health channel 有事件时才重新发送完整列表
Unhealthy是不可逆终态（代码中 FIXME 标注），设备一旦标记为不健康就无法恢复stopchannel 关闭时优雅退出 stream
3.13 设备分配：Allocate()
这是设备插件最核心的 gRPC 方法------当 Pod 请求 GPU 资源时，kubelet 调用此方法获取分配信息。
go
func (plugin *nvidiaDevicePlugin) Allocate(ctx context.Context,
reqs *pluginapi.AllocateRequest) (*pluginapi.AllocateResponse, error) {
responses := pluginapi.AllocateResponse{}
for _, req := range reqs.ContainerRequests {
// 1. 校验请求合法性
if err := plugin.rm.ValidateRequest(req.DevicesIds); err != nil {
return nil, fmt.Errorf("invalid allocation request for %q: %w", plugin.rm.Resource(), err)
}
// 2. 构造分配响应
response, err := plugin.getAllocateResponse(req.DevicesIds)
if err != nil {
return nil, fmt.Errorf("failed to get allocate response: %v", err)
}
responses.ContainerResponses = append(responses.ContainerResponses, response)
}
return &responses, nil
}getAllocateResponse() --- 分配响应构造核心：
go
func (plugin *nvidiaDevicePlugin) getAllocateResponse(requestIds []string) (*pluginapi.ContainerAllocateResponse, error) {
// 1. 从带注解的设备 ID 中提取真实设备 ID
deviceIDs := plugin.uniqueDeviceIDsFromAnnotatedDeviceIDs(requestIds)
// 2. 创建空响应
response := &pluginapi.ContainerAllocateResponse{
Envs: make(map[string]string),
}
// 3. CDI 策略处理
if plugin.deviceListStrategies.AnyCDIEnabled() {
responseID := uuid.New().String()
if err := plugin.updateResponseForCDI(response, responseID, deviceIDs...); err != nil {
return nil, fmt.Errorf("failed to get allocate response for CDI: %v", err)
}
}
// 4. MPS 处理
if plugin.mps.enabled {
plugin.updateResponseForMPS(response)
}
// 5. 高级特性环境变量
if plugin.config.Flags.GDRCopyEnabled != nil && *plugin.config.Flags.GDRCopyEnabled {
response.Envs["NVIDIA_GDRCOPY"] = "enabled"
}
if plugin.config.Flags.GDSEnabled != nil && *plugin.config.Flags.GDSEnabled {
response.Envs["NVIDIA_GDS"] = "enabled"
}
if plugin.config.Flags.MOFEDEnabled != nil && *plugin.config.Flags.MOFEDEnabled {
response.Envs["NVIDIA_MOFED"] = "enabled"
}
// 6. 如果全部使用 CDI 策略，到此返回
if plugin.deviceListStrategies.AllCDIEnabled() {
return response, nil
}
// 7. 非 CDI 策略处理
if plugin.deviceListStrategies.Includes(spec.DeviceListStrategyEnvVar) {
plugin.updateResponseForDeviceListEnvVar(response, deviceIDs...)
plugin.updateResponseForImexChannelsEnvVar(response)
}
if plugin.deviceListStrategies.Includes(spec.DeviceListStrategyVolumeMounts) {
plugin.updateResponseForDeviceMounts(response, deviceIDs...)
}
if plugin.config.Flags.Plugin.PassDeviceSpecs != nil && *plugin.config.Flags.Plugin.PassDeviceSpecs {
response.Devices = append(response.Devices,
plugin.apiDeviceSpecs(*plugin.config.Flags.NvidiaDevRoot, requestIds)...)
}
return response, nil
}分配响应构造流程图：
#mermaid-svg-bB9Z4EyiC6wmG2rp{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-bB9Z4EyiC6wmG2rp .error-icon{fill:#552222;}#mermaid-svg-bB9Z4EyiC6wmG2rp .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-bB9Z4EyiC6wmG2rp .marker{fill:#333333;stroke:#333333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .marker.cross{stroke:#333333;}#mermaid-svg-bB9Z4EyiC6wmG2rp svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-bB9Z4EyiC6wmG2rp p{margin:0;}#mermaid-svg-bB9Z4EyiC6wmG2rp .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster-label text{fill:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster-label span{color:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster-label span p{background-color:transparent;}#mermaid-svg-bB9Z4EyiC6wmG2rp .label text,#mermaid-svg-bB9Z4EyiC6wmG2rp span{fill:#333;color:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .node rect,#mermaid-svg-bB9Z4EyiC6wmG2rp .node circle,#mermaid-svg-bB9Z4EyiC6wmG2rp .node ellipse,#mermaid-svg-bB9Z4EyiC6wmG2rp .node polygon,#mermaid-svg-bB9Z4EyiC6wmG2rp .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .rough-node .label text,#mermaid-svg-bB9Z4EyiC6wmG2rp .node .label text,#mermaid-svg-bB9Z4EyiC6wmG2rp .image-shape .label,#mermaid-svg-bB9Z4EyiC6wmG2rp .icon-shape .label{text-anchor:middle;}#mermaid-svg-bB9Z4EyiC6wmG2rp .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .rough-node .label,#mermaid-svg-bB9Z4EyiC6wmG2rp .node .label,#mermaid-svg-bB9Z4EyiC6wmG2rp .image-shape .label,#mermaid-svg-bB9Z4EyiC6wmG2rp .icon-shape .label{text-align:center;}#mermaid-svg-bB9Z4EyiC6wmG2rp .node.clickable{cursor:pointer;}#mermaid-svg-bB9Z4EyiC6wmG2rp .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .arrowheadPath{fill:#333333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-bB9Z4EyiC6wmG2rp .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-bB9Z4EyiC6wmG2rp .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-bB9Z4EyiC6wmG2rp .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster text{fill:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp .cluster span{color:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-bB9Z4EyiC6wmG2rp .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-bB9Z4EyiC6wmG2rp rect.text{fill:none;stroke-width:0;}#mermaid-svg-bB9Z4EyiC6wmG2rp .icon-shape,#mermaid-svg-bB9Z4EyiC6wmG2rp .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-bB9Z4EyiC6wmG2rp .icon-shape p,#mermaid-svg-bB9Z4EyiC6wmG2rp .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-bB9Z4EyiC6wmG2rp .icon-shape .label rect,#mermaid-svg-bB9Z4EyiC6wmG2rp .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-bB9Z4EyiC6wmG2rp .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-bB9Z4EyiC6wmG2rp .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-bB9Z4EyiC6wmG2rp :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
getAllocateResponse()
提取设备 ID
CDI?
updateResponseForCDI()
MPS?
updateResponseForMPS()
设置 GDRCopy/GDS/MOFED envs
AllCDI?
返回
EnvVar?
设置 NVIDIA_VISIBLE_DEVICES
VolumeMounts?
updateResponseForDeviceMounts()
PassDeviceSpecs?
apiDeviceSpecs()
3.14 CDI 响应处理：updateResponseForCDI()
go
func (plugin *nvidiaDevicePlugin) updateResponseForCDI(
response *pluginapi.ContainerAllocateResponse,
responseID string, deviceIDs ...string) error {
// 1. 构造 CDI 设备名列表
var devices []string
for _, id := range deviceIDs {
devices = append(devices, plugin.cdiHandler.QualifiedName("gpu", id))
}
// 添加 IMEX 通道
for _, channel := range plugin.imexChannels {
devices = append(devices, plugin.cdiHandler.QualifiedName("imex-channel", channel.ID))
}
// 添加额外设备
devices = append(devices, plugin.cdiHandler.AdditionalDevices()...)
if len(devices) == 0 {
return nil
}
// 2. 根据 CDI 策略生成响应
if plugin.deviceListStrategies.Includes(spec.DeviceListStrategyCDIAnnotations) {
annotations, err := plugin.getCDIDeviceAnnotations(responseID, devices...)
if err != nil {
return err
}
response.Annotations = annotations
}
if plugin.deviceListStrategies.Includes(spec.DeviceListStrategyCDICRI) {
for _, device := range devices {
cdiDevice := pluginapi.CDIDevice{Name: device}
response.CdiDevices = append(response.CdiDevices, &cdiDevice)
}
}
return nil
}CDI 设备名格式： k8s.device-plugin.nvidia.com/gpu=<UUID>
两种 CDI 策略对比：
| 策略 | 响应字段 | 消费者 |
|---|---|---|
cdi-annotations |
Annotations map |
nvidia-container-runtime 解析注解 |
cdi-cri |
CdiDevices 列表 |
CRI runtime 直接消费 |
go
func (plugin *nvidiaDevicePlugin) getCDIDeviceAnnotations(id string, devices ...string) (map[string]string, error) {
// 使用 CDI SDK 生成标准注解
annotations, err := cdiapi.UpdateAnnotations(map[string]string{}, "nvidia-device-plugin", id, devices)
// ...
// 如果使用自定义前缀，替换标准前缀
if plugin.cdiAnnotationPrefix == spec.DefaultCDIAnnotationPrefix {
return annotations, nil
}
updatedAnnotations := make(map[string]string)
for k, v := range annotations {
newKey := plugin.cdiAnnotationPrefix + strings.TrimPrefix(k, spec.DefaultCDIAnnotationPrefix)
updatedAnnotations[newKey] = v
}
return updatedAnnotations, nil
}3.15 设备 ID 处理：uniqueDeviceIDsFromAnnotatedDeviceIDs()
go
func (plugin *nvidiaDevicePlugin) uniqueDeviceIDsFromAnnotatedDeviceIDs(ids []string) []string {
var deviceIDs []string
// 根据 ID 策略提取真实设备标识
if *plugin.config.Flags.Plugin.DeviceIDStrategy == spec.DeviceIDStrategyUUID {
deviceIDs = rm.AnnotatedIDs(ids).GetIDs() // 从注解 ID 中提取 UUID
}
if *plugin.config.Flags.Plugin.DeviceIDStrategy == spec.DeviceIDStrategyIndex {
deviceIDs = plugin.rm.Devices().Subset(ids).GetIndices() // 获取设备索引
}
// 去重（同一设备可能被多次请求）
var uniqueIDs []string
seen := make(map[string]bool)
for _, id := range deviceIDs {
if seen[id] {
continue
}
seen[id] = true
uniqueIDs = append(uniqueIDs, id)
}
return uniqueIDs
}UUID vs Index 策略：
- UUID ：设备以 GPU UUID 标识（如
GPU-abc123-...），稳定但冗长 - Index ：设备以索引号标识（如
0,1,2），简洁但可能因硬件变更而变化
3.16 DeviceSpecs 生成：apiDeviceSpecs()
go
func (plugin *nvidiaDevicePlugin) apiDeviceSpecs(devRoot string, ids []string) []*pluginapi.DeviceSpec {
// 可选设备路径 --- 不存在则跳过
optional := map[string]bool{
"/dev/nvidiactl": true,
"/dev/nvidia-uvm": true,
"/dev/nvidia-uvm-tools": true,
"/dev/nvidia-modeset": true,
}
paths := plugin.rm.GetDevicePaths(ids) // 获取设备文件路径列表
var specs []*pluginapi.DeviceSpec
for _, p := range paths {
if optional[p] {
if _, err := os.Stat(p); err != nil {
continue // 可选设备不存在则跳过
}
}
spec := &pluginapi.DeviceSpec{
ContainerPath: p, // 容器内路径（固定）
HostPath: filepath.Join(devRoot, p), // 宿主机路径（拼接 devRoot）
Permissions: "rw", // 读写权限
}
specs = append(specs, spec)
}
// 添加 IMEX 通道设备
for _, channel := range plugin.imexChannels {
spec := &pluginapi.DeviceSpec{
ContainerPath: channel.Path,
HostPath: filepath.Join(devRoot, channel.Path),
Permissions: "rw",
}
specs = append(specs, spec)
}
return specs
}DeviceSpec 结构说明：
ContainerPath：设备在容器内的路径（如/dev/nvidia0）HostPath：设备在宿主机上的路径（如/run/nvidia/driver/dev/nvidia0）Permissions：设备权限，固定为rw
可选设备处理： nvidiactl、nvidia-uvm 等控制设备可能不存在于所有配置中，通过 os.Stat 检测后决定是否包含。
3.17 PreferredAllocation：GetPreferredAllocation()
go
func (plugin *nvidiaDevicePlugin) GetPreferredAllocation(ctx context.Context,
r *pluginapi.PreferredAllocationRequest) (*pluginapi.PreferredAllocationResponse, error) {
response := &pluginapi.PreferredAllocationResponse{}
for _, req := range r.ContainerRequests {
// 委托给 ResourceManager 获取推荐分配
devices, err := plugin.rm.GetPreferredAllocation(
req.AvailableDeviceIDs, // 可用设备列表
req.MustIncludeDeviceIDs, // 必须包含的设备
int(req.AllocationSize), // 需要分配的数量
)
if err != nil {
return nil, fmt.Errorf("error getting list of preferred allocation devices: %v", err)
}
resp := &pluginapi.ContainerPreferredAllocationResponse{
DeviceIDs: devices,
}
response.ContainerResponses = append(response.ContainerResponses, resp)
}
return response, nil
}设计意图： 允许插件在调度层面提供"偏好"建议。例如对于 MIG 策略，可能偏好某些 GPU 上的特定 MIG 实例。kubelet 在 scheduling hints 中使用此信息。
3.18 Volume Mounts 策略：updateResponseForDeviceMounts()
go
func (plugin *nvidiaDevicePlugin) updateResponseForDeviceMounts(
response *pluginapi.ContainerAllocateResponse, deviceIDs ...string) {
// 设置环境变量指向 volume mount 根目录
plugin.updateResponseForDeviceListEnvVar(response, deviceListAsVolumeMountsContainerPathRoot)
// 为每个设备创建一个 volume mount
for _, id := range deviceIDs {
mount := &pluginapi.Mount{
HostPath: deviceListAsVolumeMountsHostPath, // /dev/null（哑文件）
ContainerPath: filepath.Join(deviceListAsVolumeMountsContainerPathRoot, id),
}
response.Mounts = append(response.Mounts, mount)
}
// 为 IMEX 通道也创建 volume mount
for _, channel := range plugin.imexChannels {
mount := &pluginapi.Mount{
HostPath: deviceListAsVolumeMountsHostPath,
ContainerPath: filepath.Join(deviceListAsVolumeMountsContainerPathRoot, "imex", channel.ID),
}
response.Mounts = append(response.Mounts, mount)
}
}Volume Mounts 策略： HostPath=/dev/null（哑文件），ContainerPath=/var/run/nvidia-container-devices/<deviceID>。NVIDIA Container Toolkit 读取挂载路径识别要注入的设备。
3.19 MPS 选项处理 (mps.go)
go
func (o *options) getMPSOptions(resourceManager rm.ResourceManager) (mpsOptions, error) {
// 非 MPS 策略 → 返回空选项
if o.config.Sharing.SharingStrategy() != spec.SharingStrategyMPS {
return mpsOptions{}, nil
}
// MPS 不支持 MIG 设备
for _, device := range resourceManager.Devices() {
if device.IsMigDevice() {
return mpsOptions{}, errors.New("sharing using MPS is not supported for MIG devices")
}
}
m := mpsOptions{
enabled: true,
resourceName: resourceManager.Resource(),
daemon: mps.NewDaemon(resourceManager, mps.ContainerRoot),
hostRoot: mps.Root(*o.config.Flags.MpsRoot),
}
return m, nil
}
go
func (m *mpsOptions) updateReponse(response *pluginapi.ContainerAllocateResponse) {
if m == nil || !m.enabled {
return
}
// 设置 MPS 管道目录环境变量
response.Envs["CUDA_MPS_PIPE_DIRECTORY"] = m.daemon.PipeDir()
// 添加管道目录和共享内存目录的挂载
response.Mounts = append(response.Mounts,
&pluginapi.Mount{
ContainerPath: m.daemon.PipeDir(), // 容器内管道目录
HostPath: m.hostRoot.PipeDir(m.resourceName), // 宿主机管道目录
},
&pluginapi.Mount{
ContainerPath: m.daemon.ShmDir(), // 容器内共享内存目录
HostPath: m.hostRoot.ShmDir(m.resourceName), // 宿主机共享内存目录
},
)
}MPS 机制： 多容器共享同一 GPU 计算上下文。通过 CUDA_MPS_PIPE_DIRECTORY 指定管道目录，容器挂载宿主机管道目录和共享内存目录与 MPS daemon 通信。每个资源有独立路径避免冲突。
3.20 Root 路径工具 (root.go)
go
type root string
func (r root) join(parts ...string) string {
return filepath.Join(append([]string{string(r)}, parts...)...)
}
func (r root) getDevRoot() string {
if r.isDevRoot() {
return string(r) // root 本身包含 /dev 目录 → 是 dev root
}
return "/" // 否则 dev root 默认为 /
}
func (r root) isDevRoot() bool {
stat, err := os.Stat(filepath.Join(string(r), "dev"))
if err != nil {
return false
}
return stat.IsDir()
}
func (r root) tryResolveLibrary(libraryName string) string {
if r == "" || r == "/" {
return libraryName // 根路径 → 直接返回库名（依赖系统搜索路径）
}
// 在 driver root 下搜索常见库路径
librarySearchPaths := []string{
"/usr/lib64",
"/usr/lib/x86_64-linux-gnu",
"/usr/lib/aarch64-linux-gnu",
"/lib64",
"/lib/x86_64-linux-gnu",
"/lib/aarch64-linux-gnu",
}
for _, d := range librarySearchPaths {
l := r.join(d, libraryName)
resolved, err := resolveLink(l) // 解析符号链接
if err != nil {
continue
}
return resolved
}
return libraryName // 找不到 → 返回原始名称（依赖系统路径）
}Driver Root vs Dev Root 概念：
#mermaid-svg-lp7aGlmJyzpvw4Gp{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-lp7aGlmJyzpvw4Gp .error-icon{fill:#552222;}#mermaid-svg-lp7aGlmJyzpvw4Gp .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lp7aGlmJyzpvw4Gp .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .marker.cross{stroke:#333333;}#mermaid-svg-lp7aGlmJyzpvw4Gp svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lp7aGlmJyzpvw4Gp p{margin:0;}#mermaid-svg-lp7aGlmJyzpvw4Gp .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster-label text{fill:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster-label span{color:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster-label span p{background-color:transparent;}#mermaid-svg-lp7aGlmJyzpvw4Gp .label text,#mermaid-svg-lp7aGlmJyzpvw4Gp span{fill:#333;color:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .node rect,#mermaid-svg-lp7aGlmJyzpvw4Gp .node circle,#mermaid-svg-lp7aGlmJyzpvw4Gp .node ellipse,#mermaid-svg-lp7aGlmJyzpvw4Gp .node polygon,#mermaid-svg-lp7aGlmJyzpvw4Gp .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .rough-node .label text,#mermaid-svg-lp7aGlmJyzpvw4Gp .node .label text,#mermaid-svg-lp7aGlmJyzpvw4Gp .image-shape .label,#mermaid-svg-lp7aGlmJyzpvw4Gp .icon-shape .label{text-anchor:middle;}#mermaid-svg-lp7aGlmJyzpvw4Gp .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .rough-node .label,#mermaid-svg-lp7aGlmJyzpvw4Gp .node .label,#mermaid-svg-lp7aGlmJyzpvw4Gp .image-shape .label,#mermaid-svg-lp7aGlmJyzpvw4Gp .icon-shape .label{text-align:center;}#mermaid-svg-lp7aGlmJyzpvw4Gp .node.clickable{cursor:pointer;}#mermaid-svg-lp7aGlmJyzpvw4Gp .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .arrowheadPath{fill:#333333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lp7aGlmJyzpvw4Gp .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-lp7aGlmJyzpvw4Gp .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lp7aGlmJyzpvw4Gp .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster text{fill:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp .cluster span{color:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lp7aGlmJyzpvw4Gp .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-lp7aGlmJyzpvw4Gp rect.text{fill:none;stroke-width:0;}#mermaid-svg-lp7aGlmJyzpvw4Gp .icon-shape,#mermaid-svg-lp7aGlmJyzpvw4Gp .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lp7aGlmJyzpvw4Gp .icon-shape p,#mermaid-svg-lp7aGlmJyzpvw4Gp .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-lp7aGlmJyzpvw4Gp .icon-shape .label rect,#mermaid-svg-lp7aGlmJyzpvw4Gp .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lp7aGlmJyzpvw4Gp .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-lp7aGlmJyzpvw4Gp .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-lp7aGlmJyzpvw4Gp :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 分离配置
/run/nvidia/driver
(driver-root)
libnvidia-ml.so.1
/
(dev-root)
/dev
Driver Container 节点
/run/nvidia/driver
/run/nvidia/driver/dev
/run/nvidia/driver/usr/lib64/libnvidia-ml.so.1
标准节点
/
/dev
/usr/lib64/libnvidia-ml.so.1
3.21 PreStartContainer
go
func (plugin *nvidiaDevicePlugin) PreStartContainer(context.Context,
*pluginapi.PreStartContainerRequest) (*pluginapi.PreStartContainerResponse, error) {
return &pluginapi.PreStartContainerResponse{}, nil
}空实现 --- 当前版本不需要在容器启动前执行任何操作。这是 Device Plugin API 的可选方法，保留接口完整性。
3.22 Stop 与清理
go
func (plugin *nvidiaDevicePlugin) Stop() error {
if plugin == nil || plugin.server == nil {
return nil // 防御性检查：未启动的插件直接返回
}
klog.Infof("Stopping to serve '%s' on %s", plugin.rm.Resource(), plugin.socket)
plugin.server.Stop() // 停止 gRPC server
if err := os.Remove(plugin.socket); err != nil && !os.IsNotExist(err) {
return err // 删除 socket 文件（忽略"不存在"错误）
}
plugin.cleanup() // 清理运行时状态
return nil
}停止流程： server.Stop() → os.Remove(socket) → cleanup()
3.23 Dial 工具方法
go
func (plugin *nvidiaDevicePlugin) dial(unixSocketPath string, timeout time.Duration) (*grpc.ClientConn, error) {
ctx, cancel := context.WithTimeout(plugin.ctx, timeout)
defer cancel()
c, err := grpc.DialContext(ctx, unixSocketPath,
grpc.WithTransportCredentials(insecure.NewCredentials()), // 无 TLS（Unix socket）
grpc.WithBlock(), // 阻塞直到连接成功
grpc.WithContextDialer(func(ctx context.Context, addr string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, "unix", addr) // Unix socket 拨号器
}),
)
// ...
return c, nil
}用于两个场景：(1) Serve() 中验证 server 就绪；(2) Register() 中连接 kubelet。
四、完整执行流程总览
Allocate() ListAndWatch() CheckHealth() Register() Serve() Kubelet nvidiaDevicePlugin devicePluginForResource() getResourceManagers() plugin.New() GetPlugins() startPlugins() start() Allocate() ListAndWatch() CheckHealth() Register() Serve() Kubelet nvidiaDevicePlugin devicePluginForResource() getResourceManagers() plugin.New() GetPlugins() startPlugins() start() #mermaid-svg-yOXP33SNEDj0EG71{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-yOXP33SNEDj0EG71 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-yOXP33SNEDj0EG71 .error-icon{fill:#552222;}#mermaid-svg-yOXP33SNEDj0EG71 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-yOXP33SNEDj0EG71 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-yOXP33SNEDj0EG71 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-yOXP33SNEDj0EG71 .marker.cross{stroke:#333333;}#mermaid-svg-yOXP33SNEDj0EG71 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-yOXP33SNEDj0EG71 p{margin:0;}#mermaid-svg-yOXP33SNEDj0EG71 .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-yOXP33SNEDj0EG71 text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-yOXP33SNEDj0EG71 .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-yOXP33SNEDj0EG71 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-yOXP33SNEDj0EG71 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-yOXP33SNEDj0EG71 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-yOXP33SNEDj0EG71 #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-yOXP33SNEDj0EG71 .sequenceNumber{fill:white;}#mermaid-svg-yOXP33SNEDj0EG71 #sequencenumber{fill:#333;}#mermaid-svg-yOXP33SNEDj0EG71 #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-yOXP33SNEDj0EG71 .messageText{fill:#333;stroke:none;}#mermaid-svg-yOXP33SNEDj0EG71 .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-yOXP33SNEDj0EG71 .labelText,#mermaid-svg-yOXP33SNEDj0EG71 .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-yOXP33SNEDj0EG71 .loopText,#mermaid-svg-yOXP33SNEDj0EG71 .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-yOXP33SNEDj0EG71 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-yOXP33SNEDj0EG71 .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-yOXP33SNEDj0EG71 .noteText,#mermaid-svg-yOXP33SNEDj0EG71 .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-yOXP33SNEDj0EG71 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-yOXP33SNEDj0EG71 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-yOXP33SNEDj0EG71 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-yOXP33SNEDj0EG71 .actorPopupMenu{position:absolute;}#mermaid-svg-yOXP33SNEDj0EG71 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-yOXP33SNEDj0EG71 .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-yOXP33SNEDj0EG71 .actor-man circle,#mermaid-svg-yOXP33SNEDj0EG71 line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-yOXP33SNEDj0EG71 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} altnvml strategytegra strategy loop健康监控 altCDI enabled altMPS enabled altEnvVar strategy altVolumeMounts strategy altPassDevic-eSpecs 监听事件循环 altkubelet socket 重建 /SIGHUPSIGINT/SIGTERM main() c.Run(os.Args)watch.Files(kubeletSocketDir)watch.Signals(SIGHUP, SIGINT, ...)startPlugins()loadConfig()validateFlags()rm.AddDefaultResourcesToConfig()GetPlugins()resolveStrategy("auto")cdi.New()plugin.New()getResourceManagers()nvmllib.Init()rm.NewNVMLResourceManagers()rm.NewTegraResourceManagers()devicePluginForResource() per resourcegetMPSOptions()nvidiaDevicePlugin{...}cdiHandler.CreateSpecFile()p.Start(kubeletSocket)initialize()mps.waitForDaemon()Serve()net.Listen("unix", socket)RegisterDevicePluginServer()go server.Serve(sock)dial(socket) 验证就绪Register(kubeletSocket)RegisterRequest成功go rm.CheckHealth()ListAndWatch()Send(完整设备列表)health <- DeviceSend(更新后设备列表)Allocate(AllocateRequest)ValidateRequest()getAllocateResponse()updateResponseForCDI()updateResponseForMPS()updateResponseForDeviceListEnvVar()updateResponseForDeviceMounts()apiDeviceSpecs()AllocateResponsestopPlugins() → goto restartstopPlugins() → goto exit main()
五、Device Plugin API 交互时序
Health Check Kubelet Plugin Health Check Kubelet Plugin #mermaid-svg-jH1LiutSl4iOfdbK{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-jH1LiutSl4iOfdbK .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-jH1LiutSl4iOfdbK .error-icon{fill:#552222;}#mermaid-svg-jH1LiutSl4iOfdbK .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-jH1LiutSl4iOfdbK .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-jH1LiutSl4iOfdbK .marker{fill:#333333;stroke:#333333;}#mermaid-svg-jH1LiutSl4iOfdbK .marker.cross{stroke:#333333;}#mermaid-svg-jH1LiutSl4iOfdbK svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-jH1LiutSl4iOfdbK p{margin:0;}#mermaid-svg-jH1LiutSl4iOfdbK .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-jH1LiutSl4iOfdbK text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-jH1LiutSl4iOfdbK .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-jH1LiutSl4iOfdbK .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-jH1LiutSl4iOfdbK .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-jH1LiutSl4iOfdbK .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-jH1LiutSl4iOfdbK #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-jH1LiutSl4iOfdbK .sequenceNumber{fill:white;}#mermaid-svg-jH1LiutSl4iOfdbK #sequencenumber{fill:#333;}#mermaid-svg-jH1LiutSl4iOfdbK #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-jH1LiutSl4iOfdbK .messageText{fill:#333;stroke:none;}#mermaid-svg-jH1LiutSl4iOfdbK .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-jH1LiutSl4iOfdbK .labelText,#mermaid-svg-jH1LiutSl4iOfdbK .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-jH1LiutSl4iOfdbK .loopText,#mermaid-svg-jH1LiutSl4iOfdbK .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-jH1LiutSl4iOfdbK .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-jH1LiutSl4iOfdbK .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-jH1LiutSl4iOfdbK .noteText,#mermaid-svg-jH1LiutSl4iOfdbK .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-jH1LiutSl4iOfdbK .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-jH1LiutSl4iOfdbK .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-jH1LiutSl4iOfdbK .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-jH1LiutSl4iOfdbK .actorPopupMenu{position:absolute;}#mermaid-svg-jH1LiutSl4iOfdbK .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-jH1LiutSl4iOfdbK .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-jH1LiutSl4iOfdbK .actor-man circle,#mermaid-svg-jH1LiutSl4iOfdbK line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-jH1LiutSl4iOfdbK :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} loop运行期间 Register(Version, Endpoint, ResourceName)ListAndWatch()ListAndWatchResponse(Devices)GetPreferredAllocation()PreferredAllocationResponseAllocate(ContainerRequests)AllocateResponse(Envs, Mounts, Devices, Annotations, CdiDevices)PreStartContainer(Request)PreStartContainerResponse(空)health <- Device(unhealthy)ListAndWatchResponse(更新设备列表)server.Stop() → os.Remove(socket) → cleanup()
六、Allocate 响应策略矩阵
| 策略组合 | Envs | Mounts | Devices | Annotations | CdiDevices |
|---|---|---|---|---|---|
| envvar | ✅ NVIDIA_VISIBLE_DEVICES | ❌ | ❌ | ❌ | ❌ |
| volume-mounts | ✅ (容器路径根) | ✅ (/dev/null → 容器路径) | ❌ | ❌ | ❌ |
| cdi-annotations | ✅ (特性) | ❌ | ❌ | ✅ | ❌ |
| cdi-cri | ✅ (特性) | ❌ | ❌ | ❌ | ✅ |
| envvar + pass-device-specs | ✅ | ❌ | ✅ | ❌ | ❌ |
| cdi-annotations + envvar | ✅ | ❌ | ❌ | ✅ | ❌ |
七、关键设计模式与架构总结
7.1 设计模式使用
| 模式 | 使用位置 | 说明 |
|---|---|---|
| Functional Options | options.go |
With* 函数链式配置 plugin.New() |
| Factory Method | factory.go |
New() 根据策略创建不同类型插件 |
| Strategy Pattern | getResourceManagers() |
nvml/tegra 两种设备发现策略 |
| Observer Pattern | ListAndWatch() |
health channel 推送状态变更 |
| Template Method | server.go |
Start() 定义骨架，Serve/Register/CheckHealth 为步骤 |
| Adapter | nvidiaDevicePlugin |
适配 K8s Device Plugin gRPC API 与 NVIDIA RM |
| Graceful Degradation | failOnInitError=false |
NVML 失败时不崩溃，等待重试 |
7.2 并发模型
#mermaid-svg-yiTIri42RtxBrczc{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-yiTIri42RtxBrczc .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-yiTIri42RtxBrczc .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-yiTIri42RtxBrczc .error-icon{fill:#552222;}#mermaid-svg-yiTIri42RtxBrczc .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-yiTIri42RtxBrczc .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-yiTIri42RtxBrczc .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-yiTIri42RtxBrczc .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-yiTIri42RtxBrczc .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-yiTIri42RtxBrczc .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-yiTIri42RtxBrczc .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-yiTIri42RtxBrczc .marker{fill:#333333;stroke:#333333;}#mermaid-svg-yiTIri42RtxBrczc .marker.cross{stroke:#333333;}#mermaid-svg-yiTIri42RtxBrczc svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-yiTIri42RtxBrczc p{margin:0;}#mermaid-svg-yiTIri42RtxBrczc .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-yiTIri42RtxBrczc .cluster-label text{fill:#333;}#mermaid-svg-yiTIri42RtxBrczc .cluster-label span{color:#333;}#mermaid-svg-yiTIri42RtxBrczc .cluster-label span p{background-color:transparent;}#mermaid-svg-yiTIri42RtxBrczc .label text,#mermaid-svg-yiTIri42RtxBrczc span{fill:#333;color:#333;}#mermaid-svg-yiTIri42RtxBrczc .node rect,#mermaid-svg-yiTIri42RtxBrczc .node circle,#mermaid-svg-yiTIri42RtxBrczc .node ellipse,#mermaid-svg-yiTIri42RtxBrczc .node polygon,#mermaid-svg-yiTIri42RtxBrczc .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-yiTIri42RtxBrczc .rough-node .label text,#mermaid-svg-yiTIri42RtxBrczc .node .label text,#mermaid-svg-yiTIri42RtxBrczc .image-shape .label,#mermaid-svg-yiTIri42RtxBrczc .icon-shape .label{text-anchor:middle;}#mermaid-svg-yiTIri42RtxBrczc .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-yiTIri42RtxBrczc .rough-node .label,#mermaid-svg-yiTIri42RtxBrczc .node .label,#mermaid-svg-yiTIri42RtxBrczc .image-shape .label,#mermaid-svg-yiTIri42RtxBrczc .icon-shape .label{text-align:center;}#mermaid-svg-yiTIri42RtxBrczc .node.clickable{cursor:pointer;}#mermaid-svg-yiTIri42RtxBrczc .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-yiTIri42RtxBrczc .arrowheadPath{fill:#333333;}#mermaid-svg-yiTIri42RtxBrczc .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-yiTIri42RtxBrczc .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-yiTIri42RtxBrczc .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-yiTIri42RtxBrczc .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-yiTIri42RtxBrczc .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-yiTIri42RtxBrczc .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-yiTIri42RtxBrczc .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-yiTIri42RtxBrczc .cluster text{fill:#333;}#mermaid-svg-yiTIri42RtxBrczc .cluster span{color:#333;}#mermaid-svg-yiTIri42RtxBrczc div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-yiTIri42RtxBrczc .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-yiTIri42RtxBrczc rect.text{fill:none;stroke-width:0;}#mermaid-svg-yiTIri42RtxBrczc .icon-shape,#mermaid-svg-yiTIri42RtxBrczc .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-yiTIri42RtxBrczc .icon-shape p,#mermaid-svg-yiTIri42RtxBrczc .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-yiTIri42RtxBrczc .icon-shape .label rect,#mermaid-svg-yiTIri42RtxBrczc .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-yiTIri42RtxBrczc .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-yiTIri42RtxBrczc .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-yiTIri42RtxBrczc :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 启动
启动
gRPC 请求
health channel
stop channel
server.Stop()
主 goroutine: start() 事件循环
Serve() goroutine: gRPC server.Serve(sock)
CheckHealth() goroutine: rm.CheckHealth(stop, health)
gRPC handlers: ListAndWatch / Allocate / GetPreferredAllocation
7.3 生命周期状态图
#mermaid-svg-HU2ESXf6mCpPC8sR{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-HU2ESXf6mCpPC8sR .error-icon{fill:#552222;}#mermaid-svg-HU2ESXf6mCpPC8sR .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-HU2ESXf6mCpPC8sR .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-HU2ESXf6mCpPC8sR .marker{fill:#333333;stroke:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR .marker.cross{stroke:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-HU2ESXf6mCpPC8sR p{margin:0;}#mermaid-svg-HU2ESXf6mCpPC8sR defs #statediagram-barbEnd{fill:#333333;stroke:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#mermaid-svg-HU2ESXf6mCpPC8sR g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#mermaid-svg-HU2ESXf6mCpPC8sR g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#mermaid-svg-HU2ESXf6mCpPC8sR g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-HU2ESXf6mCpPC8sR g.stateGroup line{stroke:#333333;stroke-width:1;}#mermaid-svg-HU2ESXf6mCpPC8sR .transition{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-HU2ESXf6mCpPC8sR .stateGroup .composit{fill:white;border-bottom:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR .state-note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-HU2ESXf6mCpPC8sR .state-note text{fill:black;stroke:none;font-size:10px;}#mermaid-svg-HU2ESXf6mCpPC8sR .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-HU2ESXf6mCpPC8sR .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#mermaid-svg-HU2ESXf6mCpPC8sR .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-HU2ESXf6mCpPC8sR .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-HU2ESXf6mCpPC8sR .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-HU2ESXf6mCpPC8sR .edgeLabel .label text{fill:#333;}#mermaid-svg-HU2ESXf6mCpPC8sR .label div .edgeLabel{color:#333;}#mermaid-svg-HU2ESXf6mCpPC8sR .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#mermaid-svg-HU2ESXf6mCpPC8sR .node circle.state-start{fill:#333333;stroke:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR .node .fork-join{fill:#333333;stroke:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#mermaid-svg-HU2ESXf6mCpPC8sR .end-state-inner{fill:white;stroke-width:1.5;}#mermaid-svg-HU2ESXf6mCpPC8sR .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR #statediagram-barbEnd{fill:#333333;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-HU2ESXf6mCpPC8sR .cluster-label,#mermaid-svg-HU2ESXf6mCpPC8sR .nodeLabel{color:#131300;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-cluster rect.outer{rx:5px;ry:5px;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-state .divider{stroke:#9370DB;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-state .title-state{rx:5px;ry:5px;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-cluster.statediagram-cluster .inner{fill:white;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-cluster .inner{rx:0;ry:0;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-state rect.basic{rx:5px;ry:5px;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#mermaid-svg-HU2ESXf6mCpPC8sR .note-edge{stroke-dasharray:5;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-note text{fill:black;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram-note .nodeLabel{color:black;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagram .edgeLabel{color:red;}#mermaid-svg-HU2ESXf6mCpPC8sR #dependencyStart,#mermaid-svg-HU2ESXf6mCpPC8sR #dependencyEnd{fill:#333333;stroke:#333333;stroke-width:1;}#mermaid-svg-HU2ESXf6mCpPC8sR .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-HU2ESXf6mCpPC8sR :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} plugin.New()
kubelet restart / SIGHUP
Start()
Stop() → cleanup()
Start() (restart)
SIGTERM → stopPlugins()
Created
Serving
initialize()
Serve()
Register()
go CheckHealth()
Initialized
ServingGRPC
Registered
Monitoring
7.4 核心依赖关系
#mermaid-svg-Vexqt3kn5YhF9PAQ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Vexqt3kn5YhF9PAQ .error-icon{fill:#552222;}#mermaid-svg-Vexqt3kn5YhF9PAQ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Vexqt3kn5YhF9PAQ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .marker.cross{stroke:#333333;}#mermaid-svg-Vexqt3kn5YhF9PAQ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Vexqt3kn5YhF9PAQ p{margin:0;}#mermaid-svg-Vexqt3kn5YhF9PAQ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster-label text{fill:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster-label span{color:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster-label span p{background-color:transparent;}#mermaid-svg-Vexqt3kn5YhF9PAQ .label text,#mermaid-svg-Vexqt3kn5YhF9PAQ span{fill:#333;color:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .node rect,#mermaid-svg-Vexqt3kn5YhF9PAQ .node circle,#mermaid-svg-Vexqt3kn5YhF9PAQ .node ellipse,#mermaid-svg-Vexqt3kn5YhF9PAQ .node polygon,#mermaid-svg-Vexqt3kn5YhF9PAQ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .rough-node .label text,#mermaid-svg-Vexqt3kn5YhF9PAQ .node .label text,#mermaid-svg-Vexqt3kn5YhF9PAQ .image-shape .label,#mermaid-svg-Vexqt3kn5YhF9PAQ .icon-shape .label{text-anchor:middle;}#mermaid-svg-Vexqt3kn5YhF9PAQ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .rough-node .label,#mermaid-svg-Vexqt3kn5YhF9PAQ .node .label,#mermaid-svg-Vexqt3kn5YhF9PAQ .image-shape .label,#mermaid-svg-Vexqt3kn5YhF9PAQ .icon-shape .label{text-align:center;}#mermaid-svg-Vexqt3kn5YhF9PAQ .node.clickable{cursor:pointer;}#mermaid-svg-Vexqt3kn5YhF9PAQ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .arrowheadPath{fill:#333333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Vexqt3kn5YhF9PAQ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Vexqt3kn5YhF9PAQ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Vexqt3kn5YhF9PAQ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster text{fill:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ .cluster span{color:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Vexqt3kn5YhF9PAQ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Vexqt3kn5YhF9PAQ rect.text{fill:none;stroke-width:0;}#mermaid-svg-Vexqt3kn5YhF9PAQ .icon-shape,#mermaid-svg-Vexqt3kn5YhF9PAQ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Vexqt3kn5YhF9PAQ .icon-shape p,#mermaid-svg-Vexqt3kn5YhF9PAQ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Vexqt3kn5YhF9PAQ .icon-shape .label rect,#mermaid-svg-Vexqt3kn5YhF9PAQ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Vexqt3kn5YhF9PAQ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Vexqt3kn5YhF9PAQ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Vexqt3kn5YhF9PAQ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} main.go
plugin-manager.go
root.go
internal/watch
internal/plugin/factory.go
internal/rm
go-nvml
internal/cdi
internal/imex
server.go
grpc
google/uuid
mps.go
cmd/mps-control-daemon/mps
options.go
八、代码质量备注
8.1 已知技术债务
ListAndWatchFIXME ：Unhealthy状态不可恢复apiDeviceSpecsTODO ：IMEX 通道HostPath语义可能不正确grpc.DialContextdeprecation ：建议迁移到grpc.NewClient- MPS 健康检查 TODO：仅 Start 时检查一次
8.2 健壮性设计
- gRPC 崩溃重启：5 次/1 小时退避
- kubelet 重启检测：inotify 监听 socket 创建
- 防御性 nil 检查：
Stop()检查 server == nil - 优雅降级：
failOnInitError=false支持非 GPU 节点 - 错误聚合：
errors.Join聚合多插件停止错误
8.3 安全考量
- gRPC 使用
insecure.NewCredentials()（仅 Unix socket） - 不暴露网络端口，全 Unix domain socket 通信
- 设备文件权限固定
rw，无特权提升
文档版本 : v1.0 | 分析对象 : k8s-device-plugin v0.19.3 核心插件模块 | 日期: 2026-07-26