CubeSandBox配置CubeMaster CPU 超分

当前配置:

  • CubeSandbox Server:v0.7.1

  • CubeMaster:v0.7.1,构建时间为 2026-09-11

  • 在 /etc、/opt、/data/cubelet 中没有搜到 overcommit_ratio,说明当前配置可能使用了其他配置文件路径、不同的参数名称,或者默认值;不能据此认定 CPU 超分没有生效

接下来建议先定位 v0.7.1 实际加载的配置,再调整 CPU 超分,避免修改错误配置导致沙盒调度异常

1. v0.7.1 配置文件位置

CubeSandbox 官方文档给出的单机安装配置路径是:

/usr/local/services/cubetoolbox/CubeMaster/conf.yaml

CubeMaster 的 systemd 服务通常是 cube-sandbox-cubemaster.service

先执行以下命令,检查你的实际安装情况:

复制代码
# 1. 查看 CubeMaster 服务及启动参数
systemctl status cube-sandbox-cubemaster.service --no-pager
systemctl cat cube-sandbox-cubemaster.service

# 2. 检查官方常见配置路径
ls -lah /usr/local/services/cubetoolbox/CubeMaster/
grep -nE 'overcommit|quota|cpu|mem|scheduler' \
  /usr/local/services/cubetoolbox/CubeMaster/conf.yaml 2>/dev/null

# 3. 查找其他组件配置
find /usr/local/services/cubetoolbox /etc \
  -type f \( -name 'conf.yaml' -o -name '*.yaml' -o -name '*.yml' \
  -o -name 'config.toml' \) 2>/dev/null

2. 确认 CPU 超分配置

CubeSandbox 上游曾提供调度器超分参数,默认 CPU 比例为 3、内存比例为 2,并支持按实例类型单独配置

不过,这不能直接证明 v0.7.1 的单机安装包采用相同的配置结构。因此,先不要直接往 conf.yaml 添加字段,也不要贸然重启服务

另外,上游有关于并发创建时资源超额分配的公开问题:调度器使用的资源快照可能存在约 1 秒的延迟,而且 Cubelet 与 CubeMaster 的资源配额系数可能叠加。即使配置了超分比例,也不应把它当作严格的资源隔离保证

3. 下一步

请把下面两条命令的输出定位实际配置入口:

复制代码
systemctl cat cube-sandbox-cubemaster.service

sed -n '1,240p' \
  /usr/local/services/cubetoolbox/CubeMaster/conf.yaml

第二条如果提示文件不存在,也请把报错发来。配置文件可能包含数据库密码或密钥,提交前请先脱敏

我会根据你的 v0.7.1 实际配置,进一步确定如何调整 CPU 超分比例、是否还需调整 Cubelet 配额,以及修改后如何验证生效

root@VM-0-44-tencentos \~# systemctl status cube-sandbox-cubemaster.service --no-pager ● cube-sandbox-cubemaster.service - Cube Sandbox CubeMaster Loaded: loaded (/etc/systemd/system/cube-sandbox-cubemaster.service; disabled; preset: disabled) Active: active (running) since Tue 2026-10-13 16:11:54 CST; 3 days ago Main PID: 150318 (cubemaster) Tasks: 65 (limit: 461732) Memory: 209.4M (peak: 218.8M) CPU: 1h 47min 25.280s CGroup: /system.slice/cube-sandbox-cubemaster.service └─150318 /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.625ms rows:0 SELECT * FROM t_cube_snapshot WHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' AND t_cube_snapshot.deleted_at IS NULL ORDER BY t_cube_snapshot.id LIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.272ms rows:0 SELECT * FROM t_cube_snapshot WHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' AND t_cube_snapshot.deleted_at IS NULL ORDER BY t_cube_snapshot.id LIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/pausesnap/store.go:327 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.269ms rows:0 SELECT * FROM t_cube_pause_snapshot WHERE sandbox_id = '701e116bd3f547eb840a6aa81b7d6447' ORDER BY updated_at desc,t_cube_pause_snapshot.id LIMIT 1 Oct 17 11:27:36 VM-0-44-tencentos bash150318: Oct 17 11:27:36 VM-0-44-tencentos bash150318: 2026/10/17 11:27:36 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found Oct 17 11:27:36 VM-0-44-tencentos bash150318: 0.327ms rows:0 SELECT * FROM t_cube_snapshot WHERE snapshot_id = 'tpl-bb00da010cb142298f1e16aa' AND t_cube_snapshot.deleted_at IS NULL ORDER BY t_cube_snapshot.id LIMIT 1 root@VM-0-44-tencentos \~# systemctl cat cube-sandbox-cubemaster.service # /etc/systemd/system/cube-sandbox-cubemaster.service Unit Description=Cube Sandbox CubeMaster After=network-online.target cube-sandbox-mysql.service cube-sandbox-redis.service cube-sandbox-cubeops.service Wants=network-online.target cube-sandbox-mysql.service cube-sandbox-redis.service cube-sandbox-cubeops.service ConditionPathExists=/usr/local/services/cubetoolbox/.one-click.env RequiresMountsFor=/data/CubeMaster/storage PartOf=cube-sandbox-control.target Service Type=simple User=root WorkingDirectory=/usr/local/services/cubetoolbox EnvironmentFile=-/usr/local/services/cubetoolbox/.one-click.env Environment=CUBE_MASTER_CONFIG_PATH=/usr/local/services/cubetoolbox/CubeMaster/conf.yaml ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-postcheck.sh Restart=on-failure RestartSec=2s # Override short distro defaults explicitly so SIGTERM has room to drain DB # connections / flush state before SIGKILL. TimeoutStartSec=60s TimeoutStopSec=30s Install WantedBy=cube-sandbox-control.target root@VM-0-44-tencentos \~# ls -lah /usr/local/services/cubetoolbox/CubeMaster/ total 32K drwxr-xr-x 4 1001 1001 103 Sep 20 18:19 . drwxr-xr-x 25 1001 1001 4.0K Sep 22 09:58 .. drwxr-xr-x 2 1001 1001 45 Sep 11 18:11 bin -rw-r--r-- 1 1001 1001 4.8K Sep 20 18:19 conf.yaml -rw-r--r-- 1 1001 1001 5.4K Sep 11 17:56 Dockerfile -rwxr-xr-x 1 1001 1001 4.8K Sep 11 18:11 docker-install-volume-deps.sh drwxr-xr-x 2 1001 1001 4.0K Sep 20 18:00 plugin root@VM-0-44-tencentos \~# root@VM-0-44-tencentos \~# grep -nE 'overcommit|quota|cpu|mem|scheduler' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml 2>/dev/null 102:scheduler: 108: - "cpu" 109: - "mem" root@VM-0-44-tencentos \~# find /usr/local/services/cubetoolbox /etc \ -type f \( -name 'conf.yaml' -o -name '*.yaml' -o -name '*.yml' \ -o -name 'config.toml' \) 2>/dev/null /usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml /usr/local/services/cubetoolbox/Cubelet/config/config.toml /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml /usr/local/services/cubetoolbox/webui/docker-compose.yaml /usr/local/services/cubetoolbox/support/docker-compose.yaml /usr/local/services/cubetoolbox/cubeproxy/docker-compose.yaml /usr/local/services/cubetoolbox/cube-lifecycle-manager/docker-compose.yaml /usr/local/services/cubetoolbox/CubeMaster/conf.yaml /etc/containerd/config.toml

从这次输出看,已经找到关键配置入口:

  • CubeMaster 配置文件:/usr/local/services/cubetoolbox/CubeMaster/conf.yaml

  • Cubelet 静态配置:/usr/local/services/cubetoolbox/Cubelet/config/config.toml

  • Cubelet 动态配置:/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

  • CubeMaster 通过 CUBE_MASTER_CONFIG_PATH 环境变量加载 conf.yaml,服务当前正常运行。

在 CubeMaster 配置中已经定位到 scheduler:,但 grep 只显示了 cpu、mem 两个字段,还不足以判断调度器是否支持超分参数。CPU 超分可能涉及 CubeMaster 调度器分配比例和 Cubelet 实例资源配额两层配置,需要分别确认,避免重复放大资源比例

1. 先查看调度器的完整配置

执行:

复制代码
sed -n '85,145p' \
  /usr/local/services/cubetoolbox/CubeMaster/conf.yaml

再检查 Cubelet 的资源配置:

复制代码
grep -nEi -C 4 \
  'cpu|mem|quota|overcommit|resource|limit|factor|ratio' \
  /usr/local/services/cubetoolbox/Cubelet/config/config.toml \
  /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

这两组命令只读取配置,不会修改或重启服务

2. 结合 v0.7.1 官方信息,特别注意这一点

CubeSandbox v0.7.1 发布于 2026-09-11。上游公开的超分配置说明确认了 overcommit_ratio 和 overcommit_ratio_conf 这两个参数,但公开的并发创建问题也指出,CubeMaster 与 Cubelet 的资源配额可能叠加,导致实际超分高于预期

建议在修改配置前,再执行下面这条命令,确认 Cubelet 是否配置了主机 CPU、内存配额:

复制代码
grep -n -A 30 -B 5 '\[host.quota\]' \
  /usr/local/services/cubetoolbox/Cubelet/config/config.toml

如果没有匹配到,再查看文件开头及相关配置:

复制代码
sed -n '1,220p' \
  /usr/local/services/cubetoolbox/Cubelet/config/config.toml

查看:

  1. CubeMaster/conf.yaml 的第 85--145 行

  2. Cubelet/config/config.toml 中 [host.quota] 附近的内容

拿到这两段后,就可以进一步判断应该只调 CubeMaster 调度比例,还是同时调整 Cubelet 的主机配额。暂时不要直接设置成 CPU 6 倍或更高;应先明确当前实际生效的配额,避免节点内存耗尽或沙盒运行不稳定

sed -n '85,145p' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml redis: nodes: "127.0.0.1:6379" password: "ceuhvu123" db_no: 0 max_idle: 8 max_active: 32 idle_timeout: 30 max_retry: 2 # Safety TTL (seconds) for node-metric keys; refreshed on every heartbeat so # live nodes never expire, offline nodes auto-clean. <=0 disables. node_metric_ttl_sec: 600 # Sandbox proxy route TTL. Keep 0 (disabled): the route key has no refresh # path, so any TTL below the max sandbox lifetime would break live routing. # Teardown removes it via DEL. Only set > max sandbox lifetime if needed. sandbox_proxy_ttl_sec: 0 scheduler: priority_select_num: 1 metric_update_timeout: 300s local_metric_update_timeout: 300s filter: enable_filters: - "cpu" - "mem" - "template_locality" - "realtime_create_num" # Volume plugin (S3 / COS binary reference). Credentials: # CubeMaster/plugin/volume-s3.conf (written by one-click from MinIO or CUBE_S3_*) # CubeMaster/plugin/volume-cos.conf # List order sets the default driver when Volume.create() omits driver; # S3 is first so the default install (bundled MinIO) works out of the box. volume_plugins: - name: s3 type: binary binary_path: /usr/local/services/cubetoolbox/CubeMaster/plugin/cube-volume-s3 - name: cos type: binary binary_path: /usr/local/services/cubetoolbox/CubeMaster/plugin/cube-volume-cos extra_conf: allowed_host_mount_prefixes: - "/mnt/allin_data/allin/pod_log/data_share/llm-planning/" root@VM-0-44-tencentos \~# root@VM-0-44-tencentos \~# grep -nEi -C 4 \ 'cpu|mem|quota|overcommit|resource|limit|factor|ratio' \ /usr/local/services/cubetoolbox/Cubelet/config/config.toml \ /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml /usr/local/services/cubetoolbox/Cubelet/config/config.toml-26-cubetap /usr/local/services/cubetoolbox/Cubelet/config/config.toml-27- address="/data/cubelet/cubetap.sock" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-28- gid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-29- uid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:30:operation_server /usr/local/services/cubetoolbox/Cubelet/config/config.toml:31: address="/data/cubelet/cubelet-operation.sock" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-32- gid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-33- uid = 0 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-34- disable = true /usr/local/services/cubetoolbox/Cubelet/config/config.toml-35-debug -- /usr/local/services/cubetoolbox/Cubelet/config/config.toml-39- file_num = 10 # retained files (current + numbered backups) /usr/local/services/cubetoolbox/Cubelet/config/config.toml-40- file_size = "500m" # per-file roll size: 500m, 1g, 2g (binary MiB/GiB); unitless integer is MiB /usr/local/services/cubetoolbox/Cubelet/config/config.toml-41-plugins /usr/local/services/cubetoolbox/Cubelet/config/config.toml-42- plugins."io.cubelet.controller.config.v1.cubelet" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:43: # Static-only cadence for node status and resource reporting. /usr/local/services/cubetoolbox/Cubelet/config/config.toml-44- # Do not configure this in dynamicconf/conf.yaml. /usr/local/services/cubetoolbox/Cubelet/config/config.toml-45- node_status_update_frequency = "1s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-46- cubeops_addr = "http://127.0.0.1:3010" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-47- cubeops_timeout = "10m" -- /usr/local/services/cubetoolbox/Cubelet/config/config.toml-58- plugins."io.cubelet.internal.v1.cgroup" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-59- pool_size = 3000 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-60- pool_workers = 1 /usr/local/services/cubetoolbox/Cubelet/config/config.toml-61- pool_trigger_interval_in_ms = 1000 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:62: vm_cpu_overhead = "0" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:63: vm_memory_overhead_base = "42Mi" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:64: vm_memory_overhead_coefficient = 64 /usr/local/services/cubetoolbox/Cubelet/config/config.toml:65: host_cpu_overhead = "0.3" /usr/local/services/cubetoolbox/Cubelet/config/config.toml:66: host_memory_overhead_base = "20Mi" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-67- vm_snapshot_specs_config = "/usr/local/services/cubetoolbox/cube-snapshot/spec.json" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-68- snapshot_disk_dir = "/data/snapshot_pack/disks" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-69- /usr/local/services/cubetoolbox/Cubelet/config/config.toml:70: plugins."io.cubelet.internal.v1.resource-metrics" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-71- enabled = true /usr/local/services/cubetoolbox/Cubelet/config/config.toml-72- collection_interval = "5s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-73- request_timeout = "2s" /usr/local/services/cubetoolbox/Cubelet/config/config.toml-74- max_concurrent_requests = 8 -- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-5- sandbox_exec_cmd_before_exist_log_out: false /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-6- sandbox_exec_cmd_before_exist: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-7- - sh /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-8- - "/usr/local/cubetools/cube_exec_cmd_before_exist.sh" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:9: cgroup_set_memory_reparent_file: "" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-10- disable_host_cgroup: true /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-11- # Disable the legacy netfile. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-12- disable_host_netfile: true /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-13- # Default DNS servers for guest containers. -- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-22- default_dns_options: \[\] /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-23- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-24-host: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-25- scheduler_label: "default-cluster" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:26: quota: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:27: mcpu_limit: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:28: mem_limit: "" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:29: mvm_limit: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-30- creation_concurrent_num: 0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:31: # Fraction of a paused sandbox's CPU/memory quota to release back to the /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:32: # scheduler, in 0,1, applied symmetrically to CPU and memory. A paused /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:33: # sandbox is snapshotted to disk with its MicroVM shut down, so the quota is /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:34: # physically free; this ratio trades density against resume headroom: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:35: # 0.0 (default) release nothing -- paused keeps full quota, resume /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-36- # guaranteed (legacy behaviour, safe zero-value default). /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-37- # 1.0 release everything -- max density, resume best-effort and /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-38- # rejected when the node can no longer fit the sandbox. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:39: # 0 < r < 1 release r, reserve (1-r) as headroom; the reserved quota /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:40: # still shows up in QuotaCpuUsage/QuotaMemUsage, so /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:41: # pause-heavy nodes are deprioritised by the cpu/mem scoring /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:42: # factors. /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:43: paused_resource_release_ratio: 0.0 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-44- gc: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:45: code_expiration_time: "72h" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:46: image_expiration_time: "24h" /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-47- /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-48-meta_server_config: /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-49- # Endpoints are injected by the installer (one-click / prepare-compute-role) /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml-50- # or helm at deploy time. Defaults live in pkg/config/config.go (single-node).

根据上述实际配置,已经可以把 CubeSandbox v0.7.1 的几个资源参数区分开了

目前的配置判断

配置项 当前值 含义与注意事项
CubeMaster scheduler.filter cpu、mem 等 调度器启用 CPU、内存等资源筛选,不代表已经配置了超分比例
Cubelet host.quota.mcpu_limit 0 主机 CPU 配额参数;需结合 v0.7.1 源码确认 0 的具体语义
Cubelet host.quota.mem_limit 空字符串 主机内存配额参数;需确认空值的默认处理方式
host_cpu_overhead 0.3 主机 CPU 开销估算参数,不应直接理解为 CPU 超分比例
paused_resource_release_ratio 0.0 暂停沙盒时不释放其资源配额;这会影响暂停实例较多时的可调度容量

关键结论:现在不能仅靠修改 mcpu_limit 来实现 CPU 超分。 需要确认调度器的超分比例到底由哪个配置项控制,以及 Cubelet 的配额如何参与资源统计

2. 配置 CubeMaster CPU 超分比例

上游调度器支持 overcommit_ratio,其中 CPU 和内存比例分别使用 cpu_ratio、mem_ratio。默认值为 CPU 3、内存 2

可以先备份配置,再在 scheduler: 下增加该参数。下面是建议的配置示例,修改前应确认 v0.7.1 的配置解析器接受此结构

复制代码
scheduler:
  priority_select_num: 1
  metric_update_timeout: 300s
  local_metric_update_timeout: 300s

  overcommit_ratio:
    cpu_ratio: 2
    mem_ratio: 2

  filter:
    enable_filters:
      - "cpu"
      - "mem"
      - "template_locality"
      - "realtime_create_num"

这里 CPU 设为 2,表示调度器配置的 CPU 超分比例为 2 倍,不保证最终实际超分恰好是 2 倍

3. 修改前还需要确认 Cubelet 的配额行为

目前的配置是:

复制代码
host:
  quota:
    mcpu_limit: 0
    mem_limit: ""
    mvm_limit: 0
    creation_concurrent_num: 0

公开问题 #1565 提到,Cubelet 在未显式指定 CPU、内存配额时可能采用额外的默认超分系数,再与 CubeMaster 比例叠加;还存在高并发创建时调度器读取过期资源快照的问题。该问题描述的是上游 master 分支,不能直接认定你的 v0.7.1 安装一定存在完全相同的行为

因此,我建议先不要直接改动 mcpu_limit,也不要立即重启正在运行的服务

请再执行:

复制代码
# 查看 Cubelet 的服务启动参数
systemctl list-units --type=service | grep -i cubelet

# 检查配置解析器及相关默认值
grep -RniE \
  'overcommit_ratio|cpu_ratio|mem_ratio|mcpu_limit|resolveHostQuotaCPU|resolveHostQuotaMem' \
  /usr/local/services/cubetoolbox/CubeMaster \
  /usr/local/services/cubetoolbox/Cubelet \
  2>/dev/null | head -150

如果源码或配置中没有包含可读的实现(例如只有二进制文件),输出结果即可

最终目标是明确两件事: 一是如何把 CubeMaster 的 CPU 比例设置为你想要的值;二是如何避免 Cubelet 默认系数叠加,造成实际超分过高。配置完成后,还应通过小批量沙盒创建测试验证实际可调度容量,并监控主机 CPU、内存及 OOM 事件

root@VM-0-44-tencentos \~# systemctl list-units --type=service | grep -i cubelet cube-sandbox-cubelet.service loaded active running Cube Sandbox Cubelet root@VM-0-44-tencentos \~# grep -RniE \ 'overcommit_ratio|cpu_ratio|mem_ratio|mcpu_limit|resolveHostQuotaCPU|resolveHostQuotaMem' \ /usr/local/services/cubetoolbox/CubeMaster \ /usr/local/services/cubetoolbox/Cubelet \ 2>/dev/null | head -150 /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml:27: mcpu_limit: 0

这次结果说明,当前安装目录下的 CubeMaster、Cubelet 配置文件没有暴露 overcommit_ratio 相关字段,而且 mcpu_limit: 0 是目前能确认的唯一主机 CPU 配额配置

这不代表 v0.7.1 不支持 CPU 超分:相关参数可能由程序内置默认值处理,也可能使用其他配置入口。由于当前安装包中没有找到相关源码,暂时不建议直接添加未经验证的 YAML 字段

下一步:确认 CubeMaster 的真实配置格式

先查看 CubeMaster 启动脚本、配置文件结构和版本帮助:

复制代码
# 1. 查看启动脚本
sed -n '1,220p' \
  /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh

# 2. 查看完整配置文件的一级结构
grep -nE '^[[:alnum:]_-]+:' \
  /usr/local/services/cubetoolbox/CubeMaster/conf.yaml

# 3. 查看 Cubelet 启动参数
systemctl cat cube-sandbox-cubelet.service

# 4. 查看命令行帮助
/usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster --help 2>&1 | head -100

这些操作均为只读,不会影响正在运行的沙盒

3. 已核实 v0.7.1 的真实实现

进一步检查了官方 v0.7.1 标签的源码,发现一个重要区别:从 v0.7.1 开始,CubeMaster 的 scheduler.overcommit_ratio 和 overcommit_ratio_conf 已被弃用,配置会被解析,但运行时忽略。 因此,前面给出的 CubeMaster overcommit_ratio 配置示例不适用于当前版本,请不要按该示例修改

官方 Cubelet 源码还明确说明:

  • host.quota.mcpu_limit 大于 0 时,使用配置的 CPU 配额

  • mcpu_limit: 0 时,默认 CPU 配额按主机逻辑 CPU 数量的 2 倍计算

  • mem_limit 为空时,默认内存配额按主机总内存的 1.25 倍计算

所以,当前配置的 CPU 默认超分倍数是 2 倍,而不是通过 CubeMaster 的 overcommit_ratio 设置的

4. 如何设置指定的 CPU 超分比例

v0.7.1 应通过 Cubelet 的 host.quota.mcpu_limit 设置绝对 CPU 配额,单位是 millicpu:1000 等于 1 个 CPU

例如,假设主机有 32 个逻辑 CPU:

目标超分比例 mcpu_limit
1 倍 32000
1.5 倍 48000
2 倍 64000
3 倍 96000

先执行以下命令,确认主机逻辑 CPU 数量:

复制代码
nproc
lscpu | grep -E '^(CPU\(s\)|Core\(s\) per socket|Socket\(s\)|Thread\(s\) per core):'

假设 nproc 返回 32,如果目标是 1.5 倍超分,Cubelet 配置可设为:

复制代码
host:
  quota:
    mcpu_limit: 48000

注意:这是配置示例,暂时不要直接覆盖现有 host 配置块,因为还需要保留其中的 scheduler_label、mem_limit、mvm_limit 等其他设置

另外,设置更高的 CPU 配额只是允许调度器分配更多虚拟 CPU,不代表主机拥有更多物理算力;实际负载过高仍会产生 CPU 争用

请先把 nproc 的输出发给我,并告诉我希望设置为 1.5 倍、2 倍还是 3 倍。我可以据此给出精确的 mcpu_limit 数值和保留原配置的修改命令

root@VM-0-44-tencentos \~# sed -n '1,220p' \ /usr/local/services/cubetoolbox/scripts/systemd/cubemaster-start.sh #!/usr/bin/env bash # SPDX-License-Identifier: Apache-2.0 # Copyright (C) 2026 Tencent. All rights reserved. set -euo pipefail SCRIPT_DIR="(cd "(dirname "{BASH_SOURCE\[0\]}")" \&\& pwd)" # shellcheck source=./common.sh source "{SCRIPT_DIR}/common.sh" require_root ensure_systemd_runtime_dirs CUBEMASTER_BIN="{TOOLBOX_ROOT}/CubeMaster/bin/cubemaster" CUBEMASTER_CFG="{TOOLBOX_ROOT}/CubeMaster/conf.yaml" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_DEFAULT="/data/CubeMaster/storage" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_CONFIGURED="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR:-}" CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_CONFIGURED:-{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR_DEFAULT}}" ensure_executable "{CUBEMASTER_BIN}" ensure_file "{CUBEMASTER_CFG}" export CUBE_MASTER_CONFIG_PATH="{CUBEMASTER_CFG}" if mkdir -p "{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR}" \>/dev/null 2\>\&1; then export CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR="{CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR}" else log "cubemaster artifact store {CUBEMASTER_ROOTFS_ARTIFACT_STORE_DIR} unavailable, fallback handled by cubemaster" fi # CubeTemplateCenter's address. Can be set here (ad-hoc override) or in # conf.yaml as template_center_addr (persistent, per-deployment). Env wins # over yaml. The default points at the TC unit the same bundle installs on # this host, so split deployments need only edit this export or the conf.yaml # field. Export unconditionally: the binary reads it on every request, and # having it set keeps the later enable to a single edit. export CUBE_TEMPLATE_CENTER_ADDR="{CUBE_TEMPLATE_CENTER_ADDR:-http://127.0.0.1:8090}" exec "${CUBEMASTER_BIN}" root@VM-0-44-tencentos \~# grep -nE '^\[:alnum:_-]+:' \ /usr/local/services/cubetoolbox/CubeMaster/conf.yaml 1:common: 33:log: 40:cubelet_conf: 58:auth: 61:req_template_conf: 72:instance_db_config: 86:redis: 102:scheduler: 118:volume_plugins: 125:extra_conf: root@VM-0-44-tencentos \~# systemctl cat cube-sandbox-cubelet.service # /etc/systemd/system/cube-sandbox-cubelet.service Unit Description=Cube Sandbox Cubelet After=network-online.target Wants=network-online.target ConditionPathExists=/usr/local/services/cubetoolbox/.one-click.env RequiresMountsFor=/data/cubelet /data/cube-shim /data/snapshot_pack PartOf=cube-sandbox-control.target cube-sandbox-compute.target Service Type=forking ExecStartPre=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh User=root WorkingDirectory=/usr/local/services/cubetoolbox EnvironmentFile=-/usr/local/services/cubetoolbox/.one-click.env PIDFile=/run/cube-sandbox-systemd/cubelet.pid Delegate=yes KillMode=process ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh ExecStop=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-stop.sh Restart=on-failure RestartSec=2s TimeoutStartSec=60s TimeoutStopSec=30s Install WantedBy=cube-sandbox-control.target WantedBy=cube-sandbox-compute.target root@VM-0-44-tencentos \~# /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster --help 2>&1 | head -100 Usage of /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster: -v show version -version show version

这些输出进一步确认:CubeMaster 没有提供可见的命令行超分参数,启动脚本默认加载 CubeMaster/conf.yaml。接下来应聚焦 Cubelet 的配额配置,不需要再修改 CubeMaster 的 scheduler

1. 确认 Cubelet 实际加载的配置

执行:

复制代码
sed -n '1,240p' \
  /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh

sed -n '1,240p' \
  /usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh

grep -nE 'CUBELET|CONFIG|dynamicconf|config.toml' \
  /usr/local/services/cubetoolbox/.one-click.env

这一步是为了确认修改 dynamicconf/conf.yaml 后,Cubelet 是否会读取新值,以及需要怎样重启才能生效

2. 获取 CPU 数量并确定超分比例

复制代码
nproc

假设主机有 32 个逻辑 CPU,按前面核实的 mcpu_limit 配置方式计算:

目标 CPU 配额 mcpu_limit
1 倍 32000
1.5 倍 48000
2 倍 64000
3 倍 96000

计算公式:

\ \\text{mcpu\\_limit}=\\text{逻辑CPU数}\\times1000\\times\\text{目标倍数} \\

注意:这里是调度配额,不是物理 CPU 算力;如果使用容器、虚拟机或 CPU affinity 限制了 Cubelet 可用的 CPU,还要考虑实际可用资源

现在已经确认了 Cubelet 的启动方式:

复制代码
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet \
  --config /usr/local/services/cubetoolbox/Cubelet/config/config.toml \
  --dynamic-conf-path /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

也就是说,Cubelet/dynamicconf/conf.yaml 是实际传给 Cubelet 的动态配置文件,其中的 host.quota.mcpu_limit 是接下来需要重点确认的参数

另外,prepare-compute-role.sh 会在启动时处理部分配置,但从你提供的脚本看,它没有主动重写 host.quota.mcpu_limit。因此,修改前可以先备份动态配置,避免丢失现有的节点地址、配额和其他参数

现在可以确定配置修改位置

你的启动脚本明确传入了:

复制代码
--dynamic-conf-path \
  /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

而且启动前的 prepare-compute-role.sh 不会主动覆盖 host.quota.mcpu_limit。因此,修改这个文件是合理的配置方向

需要注意:CubeSandbox 上游公开的超分问题指出,Cubelet 默认 CPU 配额系数为 2 倍;但并发创建还可能因资源快照延迟而超额调度

请最后确认两个值:

复制代码
nproc
grep -n -A 8 '^host:' \
  /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

然后告诉我你希望设置的比例,例如 2 倍、3 倍或 4 倍

将按实际逻辑 CPU 数计算 mcpu_limit,并给出保留原配置的修改命令。修改后需要重启 Cubelet 才能验证新配额是否生效;建议安排维护窗口,先确认现有沙盒的运行状态,不要直接在生产高峰期重启

节点 nproc 返回 96,当前 mcpu_limit: 0

按前面讨论的配额计算方式,如果这 96 个 CPU 都是 Cubelet 实际可用的逻辑 CPU,那么目标配额如下:

目标 CPU 超分比例 mcpu_limit
1 倍 96000
1.5 倍 144000
2 倍 192000
3 倍 288000
4 倍 384000

例如,设置为 3 倍时,对应 mcpu_limit: 288000。但要注意,nproc 是当前进程可见的 CPU 数量,不一定等于主机全部逻辑 CPU;还需要确认是否存在 CPU affinity 或容器限制

你的节点有 96 个逻辑 CPU。按当前 Cubelet 的默认配额逻辑,mcpu_limit: 0 对应的默认 CPU 配额是 2 倍,即 192000 millicpu

推荐配置

如果这是用于 AI Agent 沙盒的生产节点,建议先从 2 倍开始;如果沙盒大多空闲、CPU 使用率较低,再考虑 3 倍

目标 mcpu_limit 说明
1 倍 96000 不超分
2 倍 192000 当前默认水平
3 倍 288000 更高并发,CPU 争用风险增加
4 倍 384000 需要更严格的负载监控

例如,若确定要设置 3 倍,先备份并修改配置:

复制代码
CONF=/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

# 备份
cp -a "$CONF" "${CONF}.bak.$(date +%Y%m%d%H%M%S)"

# 仅修改 mcpu_limit: 0 这一项
sed -i '/^host:/,/^[^ ]/s/^\([[:space:]]*mcpu_limit:[[:space:]]*\)0$/\1288000/' "$CONF"

# 检查修改结果
grep -n -A 8 '^host:' "$CONF"

注意:上面的修改命令只适用于已经决定设置 3 倍的情况,暂时不要执行。 还需要确认 Cubelet 是否热加载该字段,以及当前运行中的沙盒是否会受到配额变更影响。不要直接重启生产节点上的 Cubelet,以免影响正在运行的沙盒

以4倍为例

目标是将 CubeSandbox v0.7.1 节点 CPU 配额设置为 4 倍

按 nproc = 96 计算,目标值为:

\ 96 \\times 1000 \\times 4 = 384000 \\

因此,目标配置是:

复制代码
host:
  quota:
    mcpu_limit: 384000

修改文件:

/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

注意:这会设置主机 CPU 配额上限,并不代表主机获得了额外的物理 CPU 算力。上游还报告过并发创建时调度资源快照滞后、导致超额分配的风险,因此设置为 4 倍后仍需监控 CPU 争用和 OOM

1. 备份并修改配置

下面的命令会先备份原文件,再仅修改 host.quota 下的 mcpu_limit,保留其他配置

复制代码
CONF=/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml

# 备份
cp -a "$CONF" "${CONF}.bak.$(date +%Y%m%d%H%M%S)"

# 修改指定配置项
python3 - "$CONF" <<'PY'
import re
import sys
from pathlib import Path

path = Path(sys.argv[1])
text = path.read_text()

pattern = r'(?m)^host:\s*\n(?:(?!^[^\s#]).*\n)*'
match = re.search(pattern, text)
if not match:
    raise SystemExit("ERROR: 未找到 host 配置块,未修改文件")

block = match.group(0)
quota_pattern = r'(?m)^([ \t]+mcpu_limit:[ \t]*)\d+[ \t]*$'
updated, count = re.subn(quota_pattern, r'\g<1>384000', block)

if count != 1:
    raise SystemExit(
        f"ERROR: host 配置块内匹配到 {count} 个 mcpu_limit,未修改文件"
    )

path.write_text(text[:match.start()] + updated + text[match.end():])
print(f"已设置 {path}: host.quota.mcpu_limit=384000")
PY

# 检查
sed -n '24,45p' "$CONF"

预期能看到:

复制代码
host:
  scheduler_label: "default-cluster"
  quota:
    mcpu_limit: 384000

2. 让配置生效

先不要立即重启 Cubelet。 当前节点可能运行着沙盒,重启可能影响正在运行的实例

先确认 Cubelet 是否支持动态热加载:

复制代码
journalctl -u cube-sandbox-cubelet.service \
  --since "30 minutes ago" --no-pager | tail -100

如果确认必须重启,建议在维护窗口操作:

复制代码
systemctl restart cube-sandbox-cubelet.service
systemctl status cube-sandbox-cubelet.service --no-pager

重启后再检查日志,确认配置加载成功、节点重新注册且沙盒创建正常。不要仅凭服务显示 active 就认定新配额已经生效

3. 验证 CPU 超分是否合理

配置完成后建议持续观察:

复制代码
# 主机 CPU 使用率
top

# CPU 数量与可见范围
nproc
grep Cpus_allowed_list /proc/$(pgrep -xo cubelet)/status

# OOM 记录
journalctl -k --since "1 hour ago" --no-pager |
  grep -Ei 'oom|out of memory|killed process'

建议先进行小批量沙盒创建测试,再逐步提高并发。4 倍是配额目标,不是安全并发数保证;如果沙盒 CPU 使用率普遍较高,实际负载仍可能超过主机承载能力

相关推荐
Escalating_xu1 小时前
【Linux 网络编程】TCP Socket 编程(上):从 listen、accept 到完整回显通信
linux·网络·tcp/ip
黄俊懿1 小时前
【架构师从入门到进阶】第六章:服务集群优化——第八节:线程数公式
java·服务器·数据库·分布式·性能优化·架构·系统架构
老秦包你会1 小时前
Linux第十课---------Linux信号
linux
艾伦_耶格宇1 小时前
【DOCKER容器实战】-5 Docker容器镜像的构建
运维·docker·容器
是真的小外套2 小时前
2023年软考中级信息安全工程师|选择题考点+5道案例分析真题解析(含答案)
java·服务器·网络
速捷工控 PLC触摸屏还原程序2 小时前
S7-200 SMART程序上传提示密码?维修、改造和换机怎么选处理路径?
运维·服务器
鬼手点金2 小时前
Harness 工程:模型之外的竞争力
服务器·人工智能·深度学习·机器学习·bug
LMT流氓兔2 小时前
第四单元——第17课:让回答边生成、边显示
服务器·数据库·人工智能·python·oracle·langchain
北风toto2 小时前
Linux 服务器如何判断 CPU 是否支持 x86-64-v2 指令集
java·linux·服务器