当前配置:
-
CubeSandbox Server:v0.7.1
-
CubeMaster:v0.7.1,构建时间为 2026-09-11
-
在
/etc、/opt、/data/cubelet中没有搜到overcommit_ratio,说明当前配置可能使用了其他配置文件路径、不同的参数名称,或者默认值;不能据此认定 CPU 超分没有生效
接下来建议先定位 v0.7.1 实际加载的配置,再调整 CPU 超分,避免修改错误配置导致沙盒调度异常
1. v0.7.1 配置文件位置
CubeSandbox 官方文档给出的单机安装配置路径是:
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml
CubeMaster 的 systemd 服务通常是 cube-sandbox-cubemaster.service
先执行以下命令,检查你的实际安装情况:
# 1. 查看 CubeMaster 服务及启动参数
systemctl status cube-sandbox-cubemaster.service --no-pager
systemctl cat cube-sandbox-cubemaster.service
# 2. 检查官方常见配置路径
ls -lah /usr/local/services/cubetoolbox/CubeMaster/
grep -nE 'overcommit|quota|cpu|mem|scheduler' \
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml 2>/dev/null
# 3. 查找其他组件配置
find /usr/local/services/cubetoolbox /etc \
-type f \( -name 'conf.yaml' -o -name '*.yaml' -o -name '*.yml' \
-o -name 'config.toml' \) 2>/dev/null
2. 确认 CPU 超分配置
CubeSandbox 上游曾提供调度器超分参数,默认 CPU 比例为 3、内存比例为 2,并支持按实例类型单独配置
不过,这不能直接证明 v0.7.1 的单机安装包采用相同的配置结构。因此,先不要直接往 conf.yaml 添加字段,也不要贸然重启服务
另外,上游有关于并发创建时资源超额分配的公开问题:调度器使用的资源快照可能存在约 1 秒的延迟,而且 Cubelet 与 CubeMaster 的资源配额系数可能叠加。即使配置了超分比例,也不应把它当作严格的资源隔离保证
3. 下一步
请把下面两条命令的输出定位实际配置入口:
systemctl cat cube-sandbox-cubemaster.service
sed -n '1,240p' \
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml
第二条如果提示文件不存在,也请把报错发来。配置文件可能包含数据库密码或密钥,提交前请先脱敏
我会根据你的 v0.7.1 实际配置,进一步确定如何调整 CPU 超分比例、是否还需调整 Cubelet 配额,以及修改后如何验证生效
root@VM-0-44-tencentos \~# systemctl status cube-sandbox-cubemaster.service --no-pager ● cube-sandbox-cubemaster.service - Cube Sandbox CubeMaster Loaded: loaded (/etc/systemd/system/cube-sandbox-cubemaster.service; disabled; preset: disabled) Active: active (running) since Tue 2026-10-13 16:11:54 CST; 3 days ago Main PID: 150318 (cubemaster) Tasks: 65 (limit: 461732) Memory: 209.4M (peak: 218.8M) CPU: 1h 47min 25.280s CGroup: /system.slice/cube-sandbox-cubemaster.service └─150318 /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.625ms rows:0 SELECT * FROM
t_cube_snapshotWHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.272ms rows:0 SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/pausesnap/store.go:327 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.269ms rows:0 SELECT * FROMt_cube_pause_snapshotWHERE sandbox_id = '701e116bd3f547eb840a6aa81b7d6447' ORDER BY updated_at desc,t_cube_pause_snapshot.idLIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.327ms rows:0 SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 root@VM-0-44-tencentos \~# systemctl cat cube-sandbox-cubemaster.service # /etc/systemd/system/cube-sandbox-cubemaster.service Unit Description=Cube Sandbox CubeMaster After=network-online.target cube-sandbox-mysql.service cube-sandbox-redis.service cube-sandbox-cubeops.service Wants=network-online.target cube-sandbox-mysql.service cube-sandbox-redis.service cube-sandbox-cubeops.service ConditionPathExists=/usr/local/services/cubetoolbox/.one-click.env RequiresMountsFor=/data/CubeMaster/storage PartOf=cube-sandbox-control.target Service Type=simple User=root WorkingDirectory=/usr/local/services/cubetoolbox EnvironmentFile=-/usr/local/services/cubetoolbox/.one-click.env Environment=CUBE_MASTER_CONFIG_PATH=/usr/local/services/cubetoolbox/CubeMaster/conf.yaml ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-postcheck.sh Restart=on-failure RestartSec=2s # Override short distro defaults explicitly so SIGTERM has room to drain DB # connections / flush state before SIGKILL. TimeoutStartSec=60s TimeoutStopSec=30s Install WantedBy=cube-sandbox-control.target root@VM-0-44-tencentos \~# ls -lah /usr/local/services/cubetoolbox/CubeMaster/ total 32K drwxr-xr-x 4 1001 1001 103 Sep 20 18:19 . drwxr-xr-x 25 1001 1001 4.0K Sep 22 09:58 .. drwxr-xr-x 2 1001 1001 45 Sep 11 18:11 bin -rw-r--r-- 1 1001 1001 4.8K Sep 20 18:19 conf.yaml -rw-r--r-- 1 1001 1001 5.4K Sep 11 17:56 Dockerfile -rwxr-xr-x 1 1001 1001 4.8K Sep 11 18:11 docker-install-volume-deps.sh drwxr-xr-x 2 1001 1001 4.0K Sep 20 18:00 plugin root@VM-0-44-tencentos \~# root@VM-0-44-tencentos \~# grep -nE 'overcommit|quota|cpu|mem|scheduler' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml 2>/dev/null 102:scheduler: 108: - "cpu" 109: - "mem" root@VM-0-44-tencentos \~# find /usr/local/services/cubetoolbox /etc \ -type f \( -name 'conf.yaml' -o -name '*.yaml' -o -name '*.yml' \ -o -name 'config.toml' \) 2>/dev/null /usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml /usr/local/services/cubetoolbox/Cubelet/config/config.toml /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml /usr/local/services/cubetoolbox/webui/docker-compose.yaml /usr/local/services/cubetoolbox/support/docker-compose.yaml /usr/local/services/cubetoolbox/cubeproxy/docker-compose.yaml /usr/local/services/cubetoolbox/cube-lifecycle-manager/docker-compose.yaml /usr/local/services/cubetoolbox/CubeMaster/conf.yaml /etc/containerd/config.toml
从这次输出看,已经找到关键配置入口:
-
CubeMaster 配置文件:
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml -
Cubelet 静态配置:
/usr/local/services/cubetoolbox/Cubelet/config/config.toml -
Cubelet 动态配置:
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml -
CubeMaster 通过
CUBE_MASTER_CONFIG_PATH环境变量加载conf.yaml,服务当前正常运行。
在 CubeMaster 配置中已经定位到 scheduler:,但 grep 只显示了 cpu、mem 两个字段,还不足以判断调度器是否支持超分参数。CPU 超分可能涉及 CubeMaster 调度器分配比例和 Cubelet 实例资源配额两层配置,需要分别确认,避免重复放大资源比例
1. 先查看调度器的完整配置
执行:
sed -n '85,145p' \
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml
再检查 Cubelet 的资源配置:
grep -nEi -C 4 \
'cpu|mem|quota|overcommit|resource|limit|factor|ratio' \
/usr/local/services/cubetoolbox/Cubelet/config/config.toml \
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
这两组命令只读取配置,不会修改或重启服务
2. 结合 v0.7.1 官方信息,特别注意这一点
CubeSandbox v0.7.1 发布于 2026-09-11。上游公开的超分配置说明确认了 overcommit_ratio 和 overcommit_ratio_conf 这两个参数,但公开的并发创建问题也指出,CubeMaster 与 Cubelet 的资源配额可能叠加,导致实际超分高于预期
建议在修改配置前,再执行下面这条命令,确认 Cubelet 是否配置了主机 CPU、内存配额:
grep -n -A 30 -B 5 '\[host.quota\]' \
/usr/local/services/cubetoolbox/Cubelet/config/config.toml
如果没有匹配到,再查看文件开头及相关配置:
sed -n '1,220p' \
/usr/local/services/cubetoolbox/Cubelet/config/config.toml
查看:
-
CubeMaster/conf.yaml的第 85--145 行 -
Cubelet/config/config.toml中[host.quota]附近的内容
拿到这两段后,就可以进一步判断应该只调 CubeMaster 调度比例,还是同时调整 Cubelet 的主机配额。暂时不要直接设置成 CPU 6 倍或更高;应先明确当前实际生效的配额,避免节点内存耗尽或沙盒运行不稳定
sed -n '85,145p' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml redis: nodes: "127.0.0.1:6379" password: "ceuhvu123" db_no: 0 max_idle: 8 max_active: 32 idle_timeout: 30 max_retry: 2 # Safety TTL (seconds) for node-metric keys; refreshed on every heartbeat so # live nodes never expire, offline nodes auto-clean. <=0 disables. node_metric_ttl_sec: 600 # Sandbox proxy route TTL. Keep 0 (disabled): the route key has no refresh # path, so any TTL below the max sandbox lifetime would break live routing. # Teardown removes it via DEL. Only set > max sandbox lifetime if needed. sandbox_proxy_ttl_sec: 0 scheduler: priority_select_num: 1 metric_update_timeout: 300s local_metric_update_timeout: 300s filter: enable_filters: - "cpu" - "mem" - "template_locality" - "realtime_create_num" # Volume plugin (S3 / COS binary reference). Credentials: # CubeMaster/plugin/volume-s3.conf (written by one-click from MinIO or CUBE_S3_*) # CubeMaster/plugin/volume-cos.conf # List order sets the default driver when Volume.create() omits
driver; # S3 is first so the default install (bundled MinIO) works out of the box. volume_plugins: - name: s3 type: binary binary_path: /usr/local/services/cubetoolbox/CubeMaster/plugin/cube-volume-s3 - name: cos type: binary binary_path: /usr/local/services/cubetoolbox/CubeMaster/plugin/cube-volume-cos extra_conf: allowed_host_mount_prefixes: - "/mnt/allin_data/allin/pod_log/data_share/llm-planning/" root@VM-0-44-tencentos \~# root@VM-0-44-tencentos \~# grep -nEi -C 4 \ 'cpu|mem|quota|overcommit|resource|limit|factor|ratio' \ /usr/local/services/cubetoolbox/Cubelet/config/config.toml \ /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml /usr/local/services/cubetoolbox/Cubelet/config/config.toml-26-cubetap /usr/local/services/cubetoolbox/Cubelet/config/config.toml-27- address="/data/cubelet/cubetap.sock" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-28- gid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-29- uid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:30:operation_server /usr/local/services/cubetoolbox/Cubelet/config/config.toml:31: address="/data/cubelet/cubelet-operation.sock" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-32- gid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-33- uid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-34- disable = true /usr/local/services/cubetoolbox/Cubelet/config/config.toml-35-debug -- /usr/local/services/cubetoolbox/Cubelet/config/config.toml-39- file_num = 10 # retained files (current + numbered backups) /usr/local/services/cubetoolbox/Cubelet/config/config.toml-40- file_size = "500m" # per-file roll size: 500m, 1g, 2g (binary MiB/GiB); unitless integer is MiB /usr/local/services/cubetoolbox/Cubelet/config/config.toml-41-plugins /usr/local/services/cubetoolbox/Cubelet/config/config.toml-42- plugins."io.cubelet.controller.config.v1.cubelet" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:43: # Static-only cadence for node status and resource reporting. /usr/local/services/cubetoolbox/Cubelet/config/config.toml-44- # Do not configure this in dynamicconf/conf.yaml. /usr/local/services/cubetoolbox/Cubelet/config/config.toml-45- node_status_update_frequency = "1s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-46- cubeops_addr = "http://127.0.0.1:3010" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-47- cubeops_timeout = "10m" -- /usr/local/services/cubetoolbox/Cubelet/config/config.toml-58- plugins."io.cubelet.internal.v1.cgroup" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-59- pool_size = 3000 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-60- pool_workers = 1 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-61- pool_trigger_interval_in_ms = 1000 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:62: vm_cpu_overhead = "0" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:63: vm_memory_overhead_base = "42Mi" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:64: vm_memory_overhead_coefficient = 64 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:65: host_cpu_overhead = "0.3" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:66: host_memory_overhead_base = "20Mi" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-67- vm_snapshot_specs_config = "/usr/local/services/cubetoolbox/cube-snapshot/spec.json" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-68- snapshot_disk_dir = "/data/snapshot_pack/disks" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-69- /usr/local/services/cubetoolbox/Cubelet/config/config.toml:70: plugins."io.cubelet.internal.v1.resource-metrics" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-71- enabled = true /usr/local/services/cubetoolbox/Cubelet/config/config.toml-72- collection_interval = "5s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-73- request_timeout = "2s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-74- max_concurrent_requests = 8 -- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-5- sandbox_exec_cmd_before_exist_log_out: false /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-6- sandbox_exec_cmd_before_exist: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-7- - sh /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-8- - "/usr/local/cubetools/cube_exec_cmd_before_exist.sh" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:9: cgroup_set_memory_reparent_file: "" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-10- disable_host_cgroup: true /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-11- # Disable the legacy netfile. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-12- disable_host_netfile: true /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-13- # Default DNS servers for guest containers. -- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-22- default_dns_options: \[\] /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-23- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-24-host: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-25- scheduler_label: "default-cluster" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:26: quota: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:27: mcpu_limit: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:28: mem_limit: "" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:29: mvm_limit: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-30- creation_concurrent_num: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:31: # Fraction of a paused sandbox's CPU/memory quota to release back to the /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:32: # scheduler, in 0,1, applied symmetrically to CPU and memory. A paused /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:33: # sandbox is snapshotted to disk with its MicroVM shut down, so the quota is /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:34: # physically free; this ratio trades density against resume headroom: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:35: # 0.0 (default) release nothing -- paused keeps full quota, resume /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-36- # guaranteed (legacy behaviour, safe zero-value default). /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-37- # 1.0 release everything -- max density, resume best-effort and /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-38- # rejected when the node can no longer fit the sandbox. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:39: # 0 < r < 1 release r, reserve (1-r) as headroom; the reserved quota /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:40: # still shows up in QuotaCpuUsage/QuotaMemUsage, so /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:41: # pause-heavy nodes are deprioritised by the cpu/mem scoring /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:42: # factors. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:43: paused_resource_release_ratio: 0.0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-44- gc: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:45: code_expiration_time: "72h" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:46: image_expiration_time: "24h" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-47- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-48-meta_server_config: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-49- # Endpoints are injected by the installer (one-click / prepare-compute-role) /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-50- # or helm at deploy time. Defaults live in pkg/config/config.go (single-node).
根据上述实际配置,已经可以把 CubeSandbox v0.7.1 的几个资源参数区分开了
目前的配置判断
| 配置项 | 当前值 | 含义与注意事项 |
|---|---|---|
CubeMaster scheduler.filter |
cpu、mem 等 |
调度器启用 CPU、内存等资源筛选,不代表已经配置了超分比例 |
Cubelet host.quota.mcpu_limit |
0 |
主机 CPU 配额参数;需结合 v0.7.1 源码确认 0 的具体语义 |
Cubelet host.quota.mem_limit |
空字符串 | 主机内存配额参数;需确认空值的默认处理方式 |
host_cpu_overhead |
0.3 |
主机 CPU 开销估算参数,不应直接理解为 CPU 超分比例 |
paused_resource_release_ratio |
0.0 |
暂停沙盒时不释放其资源配额;这会影响暂停实例较多时的可调度容量 |
关键结论:现在不能仅靠修改 mcpu_limit 来实现 CPU 超分。 需要确认调度器的超分比例到底由哪个配置项控制,以及 Cubelet 的配额如何参与资源统计
2. 配置 CubeMaster CPU 超分比例
上游调度器支持 overcommit_ratio,其中 CPU 和内存比例分别使用 cpu_ratio、mem_ratio。默认值为 CPU 3、内存 2
可以先备份配置,再在 scheduler: 下增加该参数。下面是建议的配置示例,修改前应确认 v0.7.1 的配置解析器接受此结构
scheduler:
priority_select_num: 1
metric_update_timeout: 300s
local_metric_update_timeout: 300s
overcommit_ratio:
cpu_ratio: 2
mem_ratio: 2
filter:
enable_filters:
- "cpu"
- "mem"
- "template_locality"
- "realtime_create_num"
这里 CPU 设为 2,表示调度器配置的 CPU 超分比例为 2 倍,不保证最终实际超分恰好是 2 倍
3. 修改前还需要确认 Cubelet 的配额行为
目前的配置是:
host:
quota:
mcpu_limit: 0
mem_limit: ""
mvm_limit: 0
creation_concurrent_num: 0
公开问题 #1565 提到,Cubelet 在未显式指定 CPU、内存配额时可能采用额外的默认超分系数,再与 CubeMaster 比例叠加;还存在高并发创建时调度器读取过期资源快照的问题。该问题描述的是上游 master 分支,不能直接认定你的 v0.7.1 安装一定存在完全相同的行为
因此,我建议先不要直接改动 mcpu_limit,也不要立即重启正在运行的服务
请再执行:
# 查看 Cubelet 的服务启动参数
systemctl list-units --type=service | grep -i cubelet
# 检查配置解析器及相关默认值
grep -RniE \
'overcommit_ratio|cpu_ratio|mem_ratio|mcpu_limit|resolveHostQuotaCPU|resolveHostQuotaMem' \
/usr/local/services/cubetoolbox/CubeMaster \
/usr/local/services/cubetoolbox/Cubelet \
2>/dev/null | head -150
如果源码或配置中没有包含可读的实现(例如只有二进制文件),输出结果即可
最终目标是明确两件事: 一是如何把 CubeMaster 的 CPU 比例设置为你想要的值;二是如何避免 Cubelet 默认系数叠加,造成实际超分过高。配置完成后,还应通过小批量沙盒创建测试验证实际可调度容量,并监控主机 CPU、内存及 OOM 事件
root@VM-0-44-tencentos \~# systemctl list-units --type=service | grep -i cubelet cube-sandbox-cubelet.service loaded active running Cube Sandbox Cubelet root@VM-0-44-tencentos \~# grep -RniE \ 'overcommit_ratio|cpu_ratio|mem_ratio|mcpu_limit|resolveHostQuotaCPU|resolveHostQuotaMem' \ /usr/local/services/cubetoolbox/CubeMaster \ /usr/local/services/cubetoolbox/Cubelet \ 2>/dev/null | head -150 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:27: mcpu_limit: 0
这次结果说明,当前安装目录下的 CubeMaster、Cubelet 配置文件没有暴露 overcommit_ratio 相关字段,而且 mcpu_limit: 0 是目前能确认的唯一主机 CPU 配额配置
这不代表 v0.7.1 不支持 CPU 超分:相关参数可能由程序内置默认值处理,也可能使用其他配置入口。由于当前安装包中没有找到相关源码,暂时不建议直接添加未经验证的 YAML 字段
下一步:确认 CubeMaster 的真实配置格式
先查看 CubeMaster 启动脚本、配置文件结构和版本帮助:
# 1. 查看启动脚本
sed -n '1,220p' \
/usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh
# 2. 查看完整配置文件的一级结构
grep -nE '^[[:alnum:]_-]+:' \
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml
# 3. 查看 Cubelet 启动参数
systemctl cat cube-sandbox-cubelet.service
# 4. 查看命令行帮助
/usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster --help 2>&1 | head -100
这些操作均为只读,不会影响正在运行的沙盒
3. 已核实 v0.7.1 的真实实现
进一步检查了官方 v0.7.1 标签的源码,发现一个重要区别:从 v0.7.1 开始,CubeMaster 的 scheduler.overcommit_ratio 和 overcommit_ratio_conf 已被弃用,配置会被解析,但运行时忽略。 因此,前面给出的 CubeMaster overcommit_ratio 配置示例不适用于当前版本,请不要按该示例修改
官方 Cubelet 源码还明确说明:
-
host.quota.mcpu_limit大于 0 时,使用配置的 CPU 配额 -
mcpu_limit: 0时,默认 CPU 配额按主机逻辑 CPU 数量的 2 倍计算 -
mem_limit为空时,默认内存配额按主机总内存的 1.25 倍计算
所以,当前配置的 CPU 默认超分倍数是 2 倍,而不是通过 CubeMaster 的 overcommit_ratio 设置的
4. 如何设置指定的 CPU 超分比例
v0.7.1 应通过 Cubelet 的 host.quota.mcpu_limit 设置绝对 CPU 配额,单位是 millicpu:1000 等于 1 个 CPU
例如,假设主机有 32 个逻辑 CPU:
| 目标超分比例 | mcpu_limit |
|---|---|
| 1 倍 | 32000 |
| 1.5 倍 | 48000 |
| 2 倍 | 64000 |
| 3 倍 | 96000 |
先执行以下命令,确认主机逻辑 CPU 数量:
nproc
lscpu | grep -E '^(CPU\(s\)|Core\(s\) per socket|Socket\(s\)|Thread\(s\) per core):'
假设 nproc 返回 32,如果目标是 1.5 倍超分,Cubelet 配置可设为:
host:
quota:
mcpu_limit: 48000
注意:这是配置示例,暂时不要直接覆盖现有 host 配置块,因为还需要保留其中的 scheduler_label、mem_limit、mvm_limit 等其他设置
另外,设置更高的 CPU 配额只是允许调度器分配更多虚拟 CPU,不代表主机拥有更多物理算力;实际负载过高仍会产生 CPU 争用
请先把 nproc 的输出发给我,并告诉我希望设置为 1.5 倍、2 倍还是 3 倍。我可以据此给出精确的 mcpu_limit 数值和保留原配置的修改命令
root@VM-0-44-tencentos \~# sed -n '1,220p' \ /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh #!/usr/bin/env bash # SPDX-License-Identifier: Apache-2.0 # Copyright (C) 2026 Tencent. All rights reserved. set -euo pipefail SCRIPT_DIR="(cd "(dirname "{BASH_SOURCE\[0\]}")" \&\& pwd)" # shellcheck source=./common.sh source "{SCRIPT_DIR}/common.sh" require_root ensure_systemd_runtime_dirs CUBEMASTER_BIN="{TOOLBOX_ROOT}/CubeMaster/bin/cubemaster" CUBEMASTER_CFG="{TOOLBOX_ROOT}/CubeMaster/conf.yaml" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_DEFAULT="/data/CubeMaster/storage" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_CONFIGURED="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR:-}" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_CONFIGURED:-{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_DEFAULT}}" ensure_executable "{CUBEMASTER_BIN}" ensure_file "{CUBEMASTER_CFG}" export CUBE_MASTER_CONFIG_PATH="{CUBEMASTER_CFG}" if mkdir -p "{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR}" \>/dev/null 2\>\&1; then export CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR}" else log "cubemaster artifact store {CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR} unavailable, fallback handled by cubemaster" fi # CubeTemplateCenter's address. Can be set here (ad-hoc override) or in # conf.yaml as template_center_addr (persistent, per-deployment). Env wins # over yaml. The default points at the TC unit the same bundle installs on # this host, so split deployments need only edit this export or the conf.yaml # field. Export unconditionally: the binary reads it on every request, and # having it set keeps the later enable to a single edit. export CUBE_TEMPLATE_CENTER_ADDR="{CUBE_TEMPLATE_CENTER_ADDR:-http://127.0.0.1:8090}" exec "${CUBEMASTER_BIN}" root@VM-0-44-tencentos \~# grep -nE '^\[:alnum:_-]+:' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml 1:common: 33:log: 40:cubelet_conf: 58:auth: 61:req_template_conf: 72:instance_db_config: 86:redis: 102:scheduler: 118:volume_plugins: 125:extra_conf: root@VM-0-44-tencentos \~# systemctl cat cube-sandbox-cubelet.service # /etc/systemd/system/cube-sandbox-cubelet.service Unit Description=Cube Sandbox Cubelet After=network-online.target Wants=network-online.target ConditionPathExists=/usr/local/services/cubetoolbox/.one-click.env RequiresMountsFor=/data/cubelet /data/cube-shim /data/snapshot_pack PartOf=cube-sandbox-control.target cube-sandbox-compute.target Service Type=forking ExecStartPre=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh User=root WorkingDirectory=/usr/local/services/cubetoolbox EnvironmentFile=-/usr/local/services/cubetoolbox/.one-click.env PIDFile=/run/cube-sandbox-systemd/cubelet.pid Delegate=yes KillMode=process ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh ExecStop=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-stop.sh Restart=on-failure RestartSec=2s TimeoutStartSec=60s TimeoutStopSec=30s Install WantedBy=cube-sandbox-control.target WantedBy=cube-sandbox-compute.target root@VM-0-44-tencentos \~# /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster --help 2>&1 | head -100 Usage of /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster: -v show version -version show version
这些输出进一步确认:CubeMaster 没有提供可见的命令行超分参数,启动脚本默认加载 CubeMaster/conf.yaml。接下来应聚焦 Cubelet 的配额配置,不需要再修改 CubeMaster 的 scheduler
1. 确认 Cubelet 实际加载的配置
执行:
sed -n '1,240p' \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh
sed -n '1,240p' \
/usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh
grep -nE 'CUBELET|CONFIG|dynamicconf|config.toml' \
/usr/local/services/cubetoolbox/.one-click.env
这一步是为了确认修改 dynamicconf/conf.yaml 后,Cubelet 是否会读取新值,以及需要怎样重启才能生效
2. 获取 CPU 数量并确定超分比例
nproc
假设主机有 32 个逻辑 CPU,按前面核实的 mcpu_limit 配置方式计算:
| 目标 CPU 配额 | mcpu_limit |
|---|---|
| 1 倍 | 32000 |
| 1.5 倍 | 48000 |
| 2 倍 | 64000 |
| 3 倍 | 96000 |
计算公式:
\ \\text{mcpu\\_limit}=\\text{逻辑CPU数}\\times1000\\times\\text{目标倍数} \\
注意:这里是调度配额,不是物理 CPU 算力;如果使用容器、虚拟机或 CPU affinity 限制了 Cubelet 可用的 CPU,还要考虑实际可用资源
现在已经确认了 Cubelet 的启动方式:
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet \
--config /usr/local/services/cubetoolbox/Cubelet/config/config.toml \
--dynamic-conf-path /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
也就是说,Cubelet/dynamicconf/conf.yaml 是实际传给 Cubelet 的动态配置文件,其中的 host.quota.mcpu_limit 是接下来需要重点确认的参数
另外,prepare-compute-role.sh 会在启动时处理部分配置,但从你提供的脚本看,它没有主动重写 host.quota.mcpu_limit。因此,修改前可以先备份动态配置,避免丢失现有的节点地址、配额和其他参数
现在可以确定配置修改位置
你的启动脚本明确传入了:
--dynamic-conf-path \
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
而且启动前的 prepare-compute-role.sh 不会主动覆盖 host.quota.mcpu_limit。因此,修改这个文件是合理的配置方向
需要注意:CubeSandbox 上游公开的超分问题指出,Cubelet 默认 CPU 配额系数为 2 倍;但并发创建还可能因资源快照延迟而超额调度
请最后确认两个值:
nproc
grep -n -A 8 '^host:' \
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
然后告诉我你希望设置的比例,例如 2 倍、3 倍或 4 倍
将按实际逻辑 CPU 数计算 mcpu_limit,并给出保留原配置的修改命令。修改后需要重启 Cubelet 才能验证新配额是否生效;建议安排维护窗口,先确认现有沙盒的运行状态,不要直接在生产高峰期重启
节点 nproc 返回 96,当前 mcpu_limit: 0
按前面讨论的配额计算方式,如果这 96 个 CPU 都是 Cubelet 实际可用的逻辑 CPU,那么目标配额如下:
| 目标 CPU 超分比例 | mcpu_limit |
|---|---|
| 1 倍 | 96000 |
| 1.5 倍 | 144000 |
| 2 倍 | 192000 |
| 3 倍 | 288000 |
| 4 倍 | 384000 |
例如,设置为 3 倍时,对应 mcpu_limit: 288000。但要注意,nproc 是当前进程可见的 CPU 数量,不一定等于主机全部逻辑 CPU;还需要确认是否存在 CPU affinity 或容器限制
你的节点有 96 个逻辑 CPU。按当前 Cubelet 的默认配额逻辑,mcpu_limit: 0 对应的默认 CPU 配额是 2 倍,即 192000 millicpu
推荐配置
如果这是用于 AI Agent 沙盒的生产节点,建议先从 2 倍开始;如果沙盒大多空闲、CPU 使用率较低,再考虑 3 倍
| 目标 | mcpu_limit |
说明 |
|---|---|---|
| 1 倍 | 96000 |
不超分 |
| 2 倍 | 192000 |
当前默认水平 |
| 3 倍 | 288000 |
更高并发,CPU 争用风险增加 |
| 4 倍 | 384000 |
需要更严格的负载监控 |
例如,若确定要设置 3 倍,先备份并修改配置:
CONF=/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
# 备份
cp -a "$CONF" "${CONF}.bak.$(date +%Y%m%d%H%M%S)"
# 仅修改 mcpu_limit: 0 这一项
sed -i '/^host:/,/^[^ ]/s/^\([[:space:]]*mcpu_limit:[[:space:]]*\)0$/\1288000/' "$CONF"
# 检查修改结果
grep -n -A 8 '^host:' "$CONF"
注意:上面的修改命令只适用于已经决定设置 3 倍的情况,暂时不要执行。 还需要确认 Cubelet 是否热加载该字段,以及当前运行中的沙盒是否会受到配额变更影响。不要直接重启生产节点上的 Cubelet,以免影响正在运行的沙盒
以4倍为例
目标是将 CubeSandbox v0.7.1 节点 CPU 配额设置为 4 倍
按 nproc = 96 计算,目标值为:
\ 96 \\times 1000 \\times 4 = 384000 \\
因此,目标配置是:
host:
quota:
mcpu_limit: 384000
修改文件:
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
注意:这会设置主机 CPU 配额上限,并不代表主机获得了额外的物理 CPU 算力。上游还报告过并发创建时调度资源快照滞后、导致超额分配的风险,因此设置为 4 倍后仍需监控 CPU 争用和 OOM
1. 备份并修改配置
下面的命令会先备份原文件,再仅修改 host.quota 下的 mcpu_limit,保留其他配置
CONF=/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
# 备份
cp -a "$CONF" "${CONF}.bak.$(date +%Y%m%d%H%M%S)"
# 修改指定配置项
python3 - "$CONF" <<'PY'
import re
import sys
from pathlib import Path
path = Path(sys.argv[1])
text = path.read_text()
pattern = r'(?m)^host:\s*\n(?:(?!^[^\s#]).*\n)*'
match = re.search(pattern, text)
if not match:
raise SystemExit("ERROR: 未找到 host 配置块,未修改文件")
block = match.group(0)
quota_pattern = r'(?m)^([ \t]+mcpu_limit:[ \t]*)\d+[ \t]*$'
updated, count = re.subn(quota_pattern, r'\g<1>384000', block)
if count != 1:
raise SystemExit(
f"ERROR: host 配置块内匹配到 {count} 个 mcpu_limit,未修改文件"
)
path.write_text(text[:match.start()] + updated + text[match.end():])
print(f"已设置 {path}: host.quota.mcpu_limit=384000")
PY
# 检查
sed -n '24,45p' "$CONF"
预期能看到:
host:
scheduler_label: "default-cluster"
quota:
mcpu_limit: 384000
2. 让配置生效
先不要立即重启 Cubelet。 当前节点可能运行着沙盒,重启可能影响正在运行的实例
先确认 Cubelet 是否支持动态热加载:
journalctl -u cube-sandbox-cubelet.service \
--since "30 minutes ago" --no-pager | tail -100
如果确认必须重启,建议在维护窗口操作:
systemctl restart cube-sandbox-cubelet.service
systemctl status cube-sandbox-cubelet.service --no-pager
重启后再检查日志,确认配置加载成功、节点重新注册且沙盒创建正常。不要仅凭服务显示 active 就认定新配额已经生效
3. 验证 CPU 超分是否合理
配置完成后建议持续观察:
# 主机 CPU 使用率
top
# CPU 数量与可见范围
nproc
grep Cpus_allowed_list /proc/$(pgrep -xo cubelet)/status
# OOM 记录
journalctl -k --since "1 hour ago" --no-pager |
grep -Ei 'oom|out of memory|killed process'
建议先进行小批量沙盒创建测试,再逐步提高并发。4 倍是配额目标,不是安全并发数保证;如果沙盒 CPU 使用率普遍较高,实际负载仍可能超过主机承载能力