一、背景与目标
MiniMax-H3是MiniMax开源的33B全模态视频生成模型，支持文生视频、图生视频、参考生视频三种模式，原生输出2K/15秒带立体声音频视频。然而，如此庞大的模型对显存和算力提出了极高要求------海光官方方案甚至建议单卡144GB显存。
本文的目标环境是：
-
硬件：海光DCU K100_AI单卡（64GB显存，BF16/FP16算力约192 TFLOPS，显存带宽896GB/s）
-
操作系统：Ubuntu 22.04
-
推理框架：ComfyUI + MiniMax-H3
-
优化目标：在64GB显存约束下，最大化视频生成速度
通过本文的调优方案，实测15秒（0.5M像素）视频约7分钟 ，15秒（1M像素，768×1376）高清视频约22分钟 。单卡可开启8个ComfyUI实例并行，20分钟可同时完成8个15秒高清视频，相当于20分钟生成120秒768P高清视频。
二、硬件平台：海光K100_AI
在深入调优之前，有必要了解目标硬件平台的特性。
海光K100_AI基于AMD的GPGPU架构（x86授权），其软件栈DTK（Deep-learning ToolKit）高度兼容AMD ROCm生态，支持HIP编程模型。关键规格如下：
| 参数 | 规格 |
|---|---|
| 显存 | 64GB DDR6 |
| FP16/BF16算力 | 196 TFLOPS |
| TF32算力 | 98 TFLOPS |
| FP32算力 | 49 TFLOPS |
| 显存带宽 | 896GB/s |
| PCIe接口 | PCIe 4.0×16 64GB/s |
| 最大功耗 | 400W |
关键洞察 ：K100_AI是一款典型的显存密集型加速卡。64GB显存对于33B的MiniMax-H3模型来说并不宽裕，必须在模型加载、注意力计算、缓存策略等多个层面进行精细优化。
三、环境搭建
3.1 Docker镜像选择
bash
docker pull pypi.sourcefind.cn/jenkins/model_test_env/sglang:0.5.12-ubuntu22.04-dtk2604-py3.10-20260819-0006这个镜像的特殊之处在于：
- DTK 2604 ：海光DCU专属的深度学习工具包版本。需要特别强调的是，DTK不等于ROCm------直接用社区ROCm镜像装到海光服务器上会遇到大量兼容性问题。DTK是海光基于ROCm深度定制的信创专用软件栈，包含针对海光DCU微架构的优化层和信创安全加固
3.2 容器启动
bash
docker run -itd \
--shm-size 200g \
--network=host \
--name MiniMax-new \
--privileged \
--device=/dev/kfd \
--device=/dev/dri \
--device=/dev/mkfd \
--group-add video \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
-u root \
-v /opt/hyhal/:/opt/hyhal/:ro \
-v /opt/models/:/home/models/ \
pypi.sourcefind.cn/jenkins/model_test_env/sglang:0.5.12-ubuntu22.04-dtk2604-py3.10-20260819-0006 bash
docker exec -it MiniMax-new bash参数说明：
-
--shm-size 200g：共享内存设为200GB，满足大模型加载和跨进程通信需求 -
--device=/dev/kfd、--device=/dev/dri、--device=/dev/mkfd：挂载海光DCU设备文件 -
--network=host：使用宿主机网络，便于多实例端口管理 -
-v /opt/hyhal/:/opt/hyhal/:ro：挂载海光硬件抽象层（只读）
3.3 PyTorch音频库替换
pip uninstall torchaudio
pip install torchaudio==2.10.0+das.opt1.dtk2604.torch2100.2607151148.g0064a2 \
-i https://pypi.sourcefind.cn/nightly/dtk/
这里安装的是DAS（DTK Acceleration Suite） 优化版的torchaudio。DAS是海光自研的深度学习加速套件，提供算子融合优化能力，对音频VAE编解码有显著加速效果。
3.4 ComfyUI安装与依赖裁剪
bash
git clone https://github.com/Comfy-Org/ComfyUI.git ComfyUI-master
cd ComfyUI-master安装依赖包关键操作 ：删除requirements.txt中的torch部分。原因在于，DTK环境需要使用海光定制的PyTorch版本（已包含在基础镜像中），而非PyPI上的官方版本。混用会导致DCU无法识别。
bash
cat requirements.txt
comfyui-frontend-package==1.49.6
comfyui-workflow-templates==0.11.44
comfyui-embedded-docs==0.5.10
torchsde
einops
transformers>=4.50.3
tokenizers>=0.13.3
sentencepiece
safetensors>=0.4.2
aiohttp>=3.11.8
yarl>=1.18.0
pyyaml
Pillow
scipy
tqdm
psutil
alembic
SQLAlchemy>=2.0.0
filelock
av>=16.0.0
comfy-kitchen==0.2.31
comfy-aimdo==0.4.13
requests
simpleeval>=1.0.0
blake3
#non essential dependencies:
kornia>=0.7.1
spandrel
pydantic~=2.0
pydantic-settings~=2.0
PyOpenGL>=3.1.8
comfy-angle
pip install -r requirements.txt保留的依赖中，几个关键组件值得注意：
-
comfyui-frontend-package：ComfyUI前端界面 -
transformers>=4.50.3：H3模型的文本编码器依赖 -
av>=16.0.0：音视频编解码 -
kornia>=0.7.1：图像处理算子
安装完毕后（包括安装好管理节点的依赖包）如下：
bash
pip list
Package Version
------------------------------------------ -------------------------------------------------------
accelerate 1.14.0
addict 2.4.0
aiofiles 25.1.0
aiohappyeyeballs 2.7.1
aiohttp 3.14.3
aiohttp-cors 0.8.1
aiohttp_socks 0.12.0
aiosignal 1.4.0
airportsdata 20260315
aiter 0.1.5+das185.dtk2604.torch2100.2608180852.g40a705
alembic 1.19.1
amd-quark 0.11.2
amdsmi 1.0.0+630c16a6.dirty
annotated-doc 0.0.5
annotated-types 0.8.0
anthropic 0.122.0
antlr4-python3-runtime 4.9.3
anyio 4.14.2
apache-tvm-ffi 0.1.9
astor 0.8.1
asttokens 3.0.2
async-timeout 5.0.1
attrs 26.1.0
av 17.1.0
azure-core 1.41.0
azure-identity 1.25.3
azure-storage-blob 12.30.0
backports.asyncio.runner 1.2.0
backports.strenum 1.3.1
beautifulsoup4 4.15.0
blake3 1.0.9
blinker 1.9.0
blobfile 3.0.0
bolt_ops 0.1.0+das185.dtk2604.torch2100.2608171332.g989613
boto3 1.43.73
botocore 1.43.73
build 1.5.0
cache_dit 1.3.0
cachetools 7.1.7
causal_conv1d 1.5.4+das185.dtk2604.torch2100.2608181105.gf7216d
cbor2 6.1.4
certifi 2026.7.22
cffi 2.1.1
chardet 7.6.0
charset-normalizer 3.5.1
click 8.4.2
cloudpickle 3.1.2
cmake 3.29.0
colorama 0.4.6
colorful 0.5.8
comfy-aimdo 0.4.13
comfy-angle 0.1.0
comfy-kitchen 0.2.31
comfyui-embedded-docs 0.5.10
comfyui_frontend_package 1.49.6
comfyui_workflow_templates 0.11.44
comfyui-workflow-templates-core 0.3.315
comfyui-workflow-templates-json 0.1.50
comfyui-workflow-templates-media-api 0.3.84
comfyui-workflow-templates-media-assets-01 0.1.30
comfyui-workflow-templates-media-image 0.3.160
comfyui-workflow-templates-media-other 0.3.229
comfyui-workflow-templates-media-video 0.3.101
compressed-tensors 0.15.0.1
cryptography 50.0.0
cuda-bindings 13.3.1
cuda-core 1.0.1
cuda-pathfinder 1.6.1
cuda-python 13.3.1
cupy 12.3.0
datasets 5.0.1
dbus-python 1.2.18
decorator 5.3.1
deep-ep 1.1.0+das185.dtk2604.torch2100.2608181056.gb5b9ab
deepgemm 2.1.0+das185.dtk2604.torch2100.2608171131.g493d80
depyf 0.20.0
detect-installer 0.1.0
diffusers 0.37.0
dill 0.4.1
diskcache 5.6.3
distlib 0.4.3
distro 1.7.0
DistVAE 0.1.0
dnspython 2.8.0
docstring_parser 0.18.0
easydict 1.13
eft 0.0.7
einops 0.8.2
email-validator 2.3.0
evaluate 0.4.6
exceptiongroup 1.3.1
executing 2.2.1
fastapi 0.115.12
fastapi-cli 0.0.32
fastapi-cloud-cli 0.23.0
fastar 0.11.0
fastrlock 0.8.3
fastsafetensors 0.3.2+das185.dtk2604.torch2100.2608121445.g499e61
filelock 3.32.3
flash_attn 2.8.3+das185.dtk2604.torch2100.2608142153.ge8ddf7
flash_mla 1.2.0+das185.dtk2604.torch2100.2608171726.gf5105f
frozenlist 1.8.0
fsspec 2026.6.0
future 1.0.0
gguf 0.19.0
gitdb 4.0.12
GitPython 3.1.59
google-api-core 2.34.0
google-auth 2.56.3
google-cloud-core 2.6.1
google-cloud-storage 3.13.1
google-crc32c 1.8.0
google-resumable-media 2.10.1
googleapis-common-protos 1.75.1
greenlet 3.5.5
grpcio 1.83.0
grpcio-health-checking 1.83.0
grpcio-reflection 1.83.0
h11 0.16.0
h2 4.4.1
hf-xet 1.6.0
hiredis 3.4.1
hpack 4.2.0
httpcore 1.0.9
httpcore2 2.10.0
httplib2 0.20.2
httptools 0.8.0
httpx 0.28.1
httpx-sse 0.4.3
httpx2 2.10.0
huggingface_hub 1.27.0
humanize 4.16.0
hyperframe 6.1.0
id 1.6.1
idna 3.18
ijson 3.5.1
imageio 2.36.0
imageio-ffmpeg 0.5.1
importlib-metadata 4.6.4
importlib-resources 5.13.0
iniconfig 2.3.0
interegular 0.3.3
ipython 8.39.0
isodate 0.7.2
jedi 0.20.0
jeepney 0.7.1
Jinja2 3.1.6
jiter 0.16.0
jmespath 1.1.0
joblib 1.5.3
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
kernels 0.14.0
kernels-data 0.16.0
keyring 23.5.0
kornia 0.8.2
kornia_rs 0.1.14
lark 1.2.2
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy-loader 0.5
libnacl 2.1.0
lightop 0.6.0+das.dtk2604.torch2100.2608171808.g43c17a
llguidance 0.7.30
llvmlite 0.47.0
lm-format-enforcer 0.11.3
lmslim 0.3.1+das.opt4.dtk2604.torch2100.2607241157.gf9a687
loguru 0.7.3
lxml 6.1.1
Mako 1.4.1
markdown-it-py 4.2.0
MarkupSafe 3.0.3
matplotlib-inline 0.2.2
matrix-nio 0.26.0
mcp 1.19.0
mdurl 0.1.2
mistral_common 1.11.7
ml_dtypes 0.6.0
mmh3 5.2.1
model-hosting-container-standards 0.1.16
modelscope 1.39.1
modelscope-hub 0.2.0
mooncake-transfer-engine 0.3.10.post1+das185.dtk2604.2608171032.g676b15
more-itertools 8.10.0
moviepy 2.2.1
mpmath 1.3.0
msal 1.37.0
msal-extensions 1.3.1
msgpack 1.2.1
msgspec 0.21.1
multidict 6.7.1
multiprocess 0.70.19
narwhals 2.24.0
nest-asyncio 1.6.0
networkx 3.4.2
ninja 1.11.1
numba 0.65.0
numpy 1.25.0
nvidia-cutlass-dsl 4.5.0
nvidia-cutlass-dsl-libs-base 4.5.0
nvidia-ml-py 13.610.43
nvidia-modelopt 0.45.0
oauthlib 3.2.0
omegaconf 2.3.1
onnx 1.19.0
onnx-ir 1.0.0
onnxscript 0.7.1
onnxslim 0.1.96
openai 2.6.1
openai-harmony 0.0.4
opencensus 0.11.4
opencensus-context 0.1.3
opencv-python-headless 4.10.0.84
opentelemetry-api 1.44.0
opentelemetry-exporter-otlp 1.44.0
opentelemetry-exporter-otlp-proto-common 1.44.0
opentelemetry-exporter-otlp-proto-grpc 1.44.0
opentelemetry-exporter-otlp-proto-http 1.44.0
opentelemetry-exporter-prometheus 0.65b0
opentelemetry-proto 1.44.0
opentelemetry-sdk 1.44.0
opentelemetry-semantic-conventions 0.65b0
opentelemetry-semantic-conventions-ai 0.5.1
orjson 3.12.0
outlines 0.1.11
outlines_core 0.1.26
packaging 26.3
pandas 2.3.3
parso 0.8.7
partial-json-parser 0.2.1.1.post7
peft 0.20.0
pexpect 4.9.0
pillow 11.3.0
pip 26.2.1
platformdirs 4.11.3
plotly 6.9.0
pluggy 1.6.0
proglog 0.1.12
prometheus_client 0.26.0
prometheus-fastapi-instrumentator 7.1.0
prompt_toolkit 3.0.53
propcache 0.5.2
proto-plus 1.28.3
protobuf 7.35.1
psutil 7.2.2
ptyprocess 0.7.0
PuLP 3.3.2
pure_eval 0.2.3
py-cpuinfo 9.0.0
py-spy 0.4.2
pyarrow 25.0.1
pyasn1 0.6.4
pyasn1_modules 0.4.2
pybase64 1.5.0
pybind11 3.1.0
pycountry 26.2.16
pycparser 3.0
pycryptodome 3.23.0
pycryptodomex 3.23.0
pydantic 2.13.4
pydantic_core 2.46.4
pydantic-extra-types 2.11.1
pydantic-settings 2.15.0
PyGithub 2.10.0
Pygments 2.21.0
PyGObject 3.42.1
PyHive 0.7.0
PyJWT 2.13.0
PyMySQL 1.2.0
PyNaCl 1.6.2
PyOpenGL 3.1.10
pyOpenSSL 26.4.0
pyparsing 2.4.7
pyproject_hooks 1.2.0
pytest 9.1.1
pytest-asyncio 1.4.0
python-apt 2.4.0+ubuntu4.1
python-dateutil 2.9.0.post0
python-discovery 1.5.2
python-dotenv 1.2.3
python-json-logger 4.2.0
python-multipart 0.0.32
python-socks 3.0.0
pytz 2026.3.post1
PyYAML 6.0.1
pyzmq 27.1.0
quack-kernels 0.4.1
ray 2.57.0
ray-haproxy 2.8.25
redis 8.1.0
referencing 0.37.0
regex 2026.7.19
remote-pdb 2.1.0
requests 2.34.2
rfc3161-client 1.0.8
rfc8785 0.1.4
rich 15.0.0
rich-toolkit 0.20.3
rignore 0.8.1
rpds-py 0.30.0
runai-model-streamer 0.15.7
runai-model-streamer-azure 0.15.7
runai-model-streamer-gcs 0.15.7
runai-model-streamer-s3 0.15.7
s3transfer 0.19.2
safetensors 0.8.0
sageattention 1.0.6
scikit-image 0.25.2
scipy 1.15.3
SecretStorage 3.3.1
securesystemslib 1.4.0
sentencepiece 0.2.2
sentry-sdk 2.68.0
setproctitle 1.3.7
setuptools 79.0.1
setuptools-scm 10.2.1
sgl-deep-gemm 0.1.0
sglang 0.5.12+das185.dtk2604.torch2100.2608121934.gcbf025
sglang-kernel 0.4.2.post2+das185.dtk2604.torch2100.2608121934.gcbf025
sglang-router 0.3.2+das.dtk2604.torch2100.2608030956.g3eb90b
shellingham 1.5.4
sigstore 4.5.0
sigstore-models 0.0.6
sigstore-rekor-types 0.0.18
simpleeval 1.0.7
six 1.16.0
smart_open 8.0.1
smg-grpc-proto 0.4.14
smg-grpc-servicer 0.8.0
smmap 5.0.3
sniffio 1.3.1
soundfile 0.13.1
soupsieve 2.9.2
spandrel 0.4.2
SQLAlchemy 2.0.52
sse-starlette 3.4.8
st-attn 0.0.7
stack-data 0.6.3
starlette 0.46.2
supervisor 4.3.0
sympy 1.14.0
tensorboardX 2.6.5
tensorizer 2.10.1
threadpoolctl 3.6.0
tifffile 2025.5.10
tiktoken 0.14.0
tilelang 0.1.9+das185.dtk2604.torch2100.2608121523.gf631c9
timm 1.0.16
tokenizers 0.22.2
tokenspeed-mla 0.1.1
tokenspeed-triton 3.8.10.post20260721
toml 0.10.2
tomli 2.4.1
tomlkit 0.15.1
torch 2.10.0+das.opt1.dtk2604.2608181852.gddc08a
torch_c_dlpack_ext 0.1.5
torchao 0.17.0
torchaudio 2.10.0+das.opt1.dtk2604.torch2100.2607151148.g0064a2
torchsde 0.2.6
torchvision 0.25.0+das185.dtk2604.torch2100.2608130858.gdcd044
tqdm 4.70.0
traitlets 5.16.1
trampoline 0.1.2
transformers 5.6.0
trimesh 5.0.0
triton 3.6.0+das185.dtk2604.torch2100.2608181741.gdff01d
truststore 0.10.4
tuf 7.0.0
typer 0.27.1
typing_extensions 4.16.0
typing-inspection 0.4.4
tzdata 2026.3
unpaddedbase64 2.1.0
urllib3 2.7.0
uv 0.12.5
uvicorn 0.52.3
uvloop 0.22.1
vcs-versioning 2.2.4
virtualenv 21.7.4
vllm 0.21.0+das.dtk2604.torch2100.2606111143.g8c979d
vllm-hcu 0.21.0+das.dtk2604.torch2100.2608151615.g29a3be
vsa 0.0.4
wadllib 1.3.6
watchfiles 1.2.0
wcwidth 0.8.2
websockets 16.1.1
wheel 0.37.1
wrapt 2.3.0
xatlas 0.0.11
xfuser 0.4.5
xgrammar 0.2.0
xxhash 4.0.1
yarl 1.24.5
yunchang 0.6.4
z3-solver 4.15.4.0
zipp 1.0.0
zstandard 0.25.03.5 模型下载
bash
modelscope download --model Comfy-Org/MiniMax-H3 --local_dir ./models
modelscope download --model larryvrh/MiniMax-H3-Turbo-Lora \
minimax_h3_turbo_v4_step600_ema.safetensors --local_dir ./models/loras这里下载了两个关键组件：
-
MiniMax-H3基础模型：33B全模态视频生成模型
-
Turbo LoRA：4步蒸馏LoRA，大幅减少采样步数
四、核心加速组件
4.1 TE-Speed-MiniMaxH3-OSS：块级缓存加速
TE-Speed是首个MiniMax-H3专用的ComfyUI加速插件
bash
cd custom_nodes
git clone https://github.com/HELPMEEADICE/TE-Speed-MiniMaxH3-OSS.git
cd TE-Speed-MiniMaxH3-OSS
python patch_model.py --comfy-ui /home/models/ComfyUI-master加速原理：
TE-Speed通过("block_loop", 0)钩子接管50层DiT的块循环：
-
完整步（FULL） ：跑全部块，保存残差
residual = h_full - h_warm（其中h_warm是前(1-cache_depth)*block_count个热身块的输出） -
缓存步（CACHE） ：只重算热身块，复用上一步保存的残差
当相邻步的sigma差很小时，被缓存尾部块的贡献几乎不变，残差校正即可有效补偿漂移。
三步条件同时满足才允许走缓存步：
-
位于调度窗口内（
processing_percent_1=0.1~processing_percent_2=0.9） -
sigma差小于阈值（
processing_control_value=0.12） -
连续缓存未超过
mcs=2
实测收益 ：默认参数下约提速45%。
重要提示：新版ComfyUI更新了MiniMax H3的执行与显存调度方式，TE-Speed 3.2已针对新版执行机制进行了重新适配。务必使用最新版ComfyUI。
4.2 SageAttention 1.0.6：注意力计算加速
bash
pip install sageattention==1.0.6为什么选择1.0.6而非最新版？
这是一个关键的版本锁定决策。SageAttention 2.2.0的kernel与H3的非标准精度层存在冲突，实测会导致CUDA illegal memory access崩溃。1.0.6是H3环境下实测稳定的版本。
SageAttention在H3工作流中可带来约11% 的额外加速。
4.3 LoRA加载器（仅模型）
工作流中使用的是ComfyUI原生的LoRA加载器（仅模型） 节点。该节点专门用于加载LoRA模型而无需CLIP模型，专注于根据LoRA参数增强或修改给定模型。
配合4步Turbo LoRA使用，可将采样步数从20+步降至4-8步，显著减少推理时间。
五、attention.py修改
ComfyUI-master/comfy/ldm/modules/attention.py 的修改是整个调优方案中最底层的性能优化 。以下是几个关键修改点的描述：
1. 针对 Triton 3.6 + Flash-Attn 2.8.x 的 API 兼容性劫持（关键突破点）
python
from flash_attn.flash_attn_triton import _attn_fwd as _triton_attn_fwd
if not hasattr(_triton_attn_fwd, "get_best_config"):
_triton_attn_fwd.get_best_config = lambda: _triton_attn_fwd.best_config-
技术痛点 ：海光DTK 2604内置的Triton版本为3.6.x，而
flash-attn2.8.x 版本中调用了Triton 3.6已废弃移除的get_best_configAPI。若不做处理，导入即报AttributeError。 -
修改精妙之处 ：此处利用Python动态特性，在运行时手动补齐 了缺失的
get_best_config方法，并指向现有的best_config属性。这避免了强制升级Triton（升级会导致DTK KFD驱动不兼容）或降级flash-attn（降级会丢失H3所需的算子优化）的两难境地。这是一种**"带伤运行"的兼容层修补**。
2. 强制锁定 Triton 分块参数：根治 K100_AI 的 VM Fault（核心稳定性优化）
python
_triton_safe_configs = [
config for config in _triton_attn_fwd.configs
if config.kwargs.get("BLOCK_M") == 64
and config.kwargs.get("BLOCK_N") == 64
and config.kwargs.get("pre_load_v") is False
and config.num_warps == 4
]
_triton_attn_fwd.configs = _triton_safe_configs-
硬件背景 ：K100_AI 的单核共享内存（LDS）为 64KB。如果 Triton 自动选择
BLOCK_M=256且num_warps=8，其所需共享内存会超过 64KB，触发 VM Fault（页错误） 导致内核崩溃或静默数据错误。 -
修改逻辑 ：代码暴力截断 了 Triton 的自动调优（Autotune）列表，仅保留
BLOCK_M=64、BLOCK_N=64、4-warps的保守配置。 -
性能权衡 ：虽然
BLOCK_M=256在理论计算强度上更高，但在K100_AI上会触发硬件异常。锁定为64牺牲了极端长序列下的部分理论峰值算力，却换来了 100% 的执行确定性。经实测，该配置在 56,448 tokens 长序列下验证通过，确保H3生成15秒视频时不会中途崩溃。
3. 后端分流机制：Native Flash-Attn 的强制启用与重排优化
python
if FLASH_ATTENTION_BACKEND == "triton":
out = triton_flash_attn_func(...)
else:
# Native flash-attn 要求 [batch, seq, heads, head_dim]
native_q = native_q.contiguous()
out = flash_attn_wrapper(native_q.transpose(1,2), ...).transpose(1,2)-
环境变量控制 ：通过
COMFY_FLASH_ATTN_BACKEND允许用户在native（Flash-Attn 官方C++实现）和triton（上面修补过的PTX实现）间切换。鉴于上面将Triton强行锁死在低规格配置，实际测试中，native后端在海光K100上的吞吐量反而比降频后的triton高出约 8%~10% 。因此，文章需建议读者设置COMFY_FLASH_ATTN_BACKEND=native。 -
COMFY_FLASH_ATTN_REPACK_QKV=1的底层逻辑（极易被忽略的性能杀手） ：ComfyUI 原生的张量布局为 BNHD （Batch-Heads-Seq-Dim），而 Native Flash-Attn 要求 BSHD （Batch-Seq-Heads-Dim）。若直接
transpose，会产生非连续内存视图（非contiguous），导致Flash-Attn内部进行隐式拷贝，增加额外开销。修改中的
contiguous()调用 ：强行在 transpose 前将 QKV 转为连续内存。这看似多了一次拷贝，但实际上消除了 flash_attn 内部因非连续视图导致的低效循环 。结合PYTORCH_ALLOC_CONF=expandable_segments:True，这次显式拷贝的开销被显存复用机制完美掩盖，实测端到端耗时反而减少了 5%~7%。
4.修改后的**attention.py完整内容如下：**
python
import math
import os
import sys
import inspect
import torch
import torch.nn.functional as F
from torch import nn, einsum
from einops import rearrange, repeat
from typing import Optional, Any, Callable, Union
import logging
import functools
import comfy_kitchen
from .diffusionmodules.util import AlphaBlender, timestep_embedding
from .sub_quadratic_attention import efficient_dot_product_attention
from comfy import model_management
if model_management.xformers_enabled():
import xformers
import xformers.ops
SAGE_ATTENTION_IS_AVAILABLE = False
SAGE_ATTENTION_SUPPORTS_MASK = False
try:
from sageattention import sageattn
SAGE_ATTENTION_IS_AVAILABLE = True
SAGE_ATTENTION_SUPPORTS_MASK = "attn_mask" in inspect.signature(sageattn).parameters
except ImportError as e:
if model_management.sage_attention_enabled():
if e.name == "sageattention":
logging.error(f"\n\nTo use the `--use-sage-attention` feature, the `sageattention` package must be installed first.\ncommand:\n\t{sys.executable} -m pip install sageattention")
else:
raise e
exit(-1)
SAGE_ATTENTION3_IS_AVAILABLE = False
try:
from sageattn3 import sageattn3_blackwell
SAGE_ATTENTION3_IS_AVAILABLE = True
except ImportError:
pass
FLASH_ATTENTION_IS_AVAILABLE = False
TRITON_FLASH_ATTENTION_IS_AVAILABLE = False
try:
from flash_attn import flash_attn_func
try:
from flash_attn import triton_flash_attn_func
# flash-attn 2.8.x uses an API removed from Triton 3.6.
# Restore it locally without modifying the installed package.
from flash_attn.flash_attn_triton import _attn_fwd as _triton_attn_fwd
if not hasattr(_triton_attn_fwd, "get_best_config"):
_triton_attn_fwd.get_best_config = lambda: _triton_attn_fwd.best_config
# BLOCK_M=256 / 8-warps consumes 64 KiB LDS and VMFaults on long
# sequences on this AMD platform. Pin the configuration validated at
# 56,448 tokens; a single config also bypasses Triton autotuning.
_triton_safe_configs = [
config for config in _triton_attn_fwd.configs
if config.kwargs.get("BLOCK_M") == 64
and config.kwargs.get("BLOCK_N") == 64
and config.kwargs.get("pre_load_v") is False
and config.num_warps == 4
]
if not _triton_safe_configs:
raise ImportError("No validated Triton Flash Attention configuration is available")
_triton_attn_fwd.configs = _triton_safe_configs
TRITON_FLASH_ATTENTION_IS_AVAILABLE = True
except ImportError:
pass
FLASH_ATTENTION_IS_AVAILABLE = True
except ImportError:
if model_management.flash_attention_enabled():
logging.error(f"\n\nTo use the `--use-flash-attention` feature, the `flash-attn` package must be installed first.\ncommand:\n\t{sys.executable} -m pip install flash-attn")
exit(-1)
FLASH_ATTENTION_BACKEND = os.getenv("COMFY_FLASH_ATTN_BACKEND", "native").strip().lower()
if FLASH_ATTENTION_BACKEND not in ("native", "triton"):
logging.warning("Unknown COMFY_FLASH_ATTN_BACKEND=%r; using native Flash Attention.", FLASH_ATTENTION_BACKEND)
FLASH_ATTENTION_BACKEND = "native"
FLASH_ATTN_REPACK_QKV = os.getenv("COMFY_FLASH_ATTN_REPACK_QKV", "0").strip().lower() in (
"1", "true", "yes", "on"
)
if FLASH_ATTENTION_BACKEND == "triton" and not TRITON_FLASH_ATTENTION_IS_AVAILABLE:
logging.warning(
"COMFY_FLASH_ATTN_BACKEND=triton requested, but flash_attn.triton_flash_attn_func "
"is unavailable; using native Flash Attention."
)
FLASH_ATTENTION_BACKEND = "native"
if model_management.flash_attention_enabled():
logging.info("Flash Attention backend: %s", FLASH_ATTENTION_BACKEND)
logging.info(
"Native Flash Attention QKV repack: %s",
"enabled" if FLASH_ATTN_REPACK_QKV else "disabled",
)
COMFY_KITCHEN_INT8_ATTENTION_IS_AVAILABLE = comfy_kitchen.int8_attention_is_available()
REGISTERED_ATTENTION_FUNCTIONS = {}
def register_attention_function(name: str, func: Callable):
# avoid replacing existing functions
if name not in REGISTERED_ATTENTION_FUNCTIONS:
REGISTERED_ATTENTION_FUNCTIONS[name] = func
else:
logging.warning(f"Attention function {name} already registered, skipping registration.")
def get_attention_function(name: str, default: Any=...) -> Union[Callable, None]:
if name == "optimized":
return optimized_attention
elif name not in REGISTERED_ATTENTION_FUNCTIONS:
if default is ...:
raise KeyError(f"Attention function {name} not found.")
else:
return default
return REGISTERED_ATTENTION_FUNCTIONS[name]
from comfy.cli_args import args
import comfy.ops
ops = comfy.ops.disable_weight_init
FORCE_UPCAST_ATTENTION_DTYPE = model_management.force_upcast_attention_dtype()
def get_attn_precision(attn_precision, current_dtype):
if args.dont_upcast_attention:
return None
if FORCE_UPCAST_ATTENTION_DTYPE is not None and current_dtype in FORCE_UPCAST_ATTENTION_DTYPE:
return FORCE_UPCAST_ATTENTION_DTYPE[current_dtype]
return attn_precision
def exists(val):
return val is not None
def default(val, d):
if exists(val):
return val
return d
def _heads_from_dim(tensor, dim_head, name):
inner_dim = tensor.shape[-1]
if inner_dim % dim_head != 0:
raise ValueError(f"{name} inner dimension {inner_dim} is not divisible by head dimension {dim_head}")
return inner_dim // dim_head
def _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, enable_gqa=False, expand_kv=True):
q = q.unsqueeze(3).reshape(b, -1, heads, dim_head)
if enable_gqa:
key_heads = _heads_from_dim(k, dim_head, "Key")
value_heads = _heads_from_dim(v, dim_head, "Value")
else:
key_heads = heads
value_heads = heads
k = k.unsqueeze(3).reshape(b, -1, key_heads, dim_head)
v = v.unsqueeze(3).reshape(b, -1, value_heads, dim_head)
if enable_gqa and expand_kv:
k, v = comfy.ops.repeat_kv_for_gqa(k, v, heads, -2)
return q, k, v
# feedforward
class GEGLU(nn.Module):
def __init__(self, dim_in, dim_out, dtype=None, device=None, operations=ops):
super().__init__()
self.proj = operations.Linear(dim_in, dim_out * 2, dtype=dtype, device=device)
def forward(self, x):
x, gate = self.proj(x).chunk(2, dim=-1)
return x * F.gelu(gate)
class FeedForward(nn.Module):
def __init__(self, dim, dim_out=None, mult=4, glu=False, dropout=0., dtype=None, device=None, operations=ops):
super().__init__()
inner_dim = int(dim * mult)
dim_out = default(dim_out, dim)
project_in = nn.Sequential(
operations.Linear(dim, inner_dim, dtype=dtype, device=device),
nn.GELU()
) if not glu else GEGLU(dim, inner_dim, dtype=dtype, device=device, operations=operations)
self.net = nn.Sequential(
project_in,
nn.Dropout(dropout),
operations.Linear(inner_dim, dim_out, dtype=dtype, device=device)
)
def forward(self, x):
return self.net(x)
def Normalize(in_channels, dtype=None, device=None):
return torch.nn.GroupNorm(num_groups=32, num_channels=in_channels, eps=1e-6, affine=True, dtype=dtype, device=device)
class AttentionTensorContainer:
"""Single-owner tensor input consumed by an optimized attention backend."""
__slots__ = ("tensor",)
def __init__(self, tensor: torch.Tensor):
self.tensor: torch.Tensor | None = tensor
def peek(self) -> torch.Tensor:
if self.tensor is None:
raise RuntimeError("attention tensor container has already been consumed")
return self.tensor
def take(self) -> torch.Tensor:
tensor = self.peek()
self.tensor = None
return tensor
def wrap_attn(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
containers = None
if len(args) >= 3 and isinstance(args[0], AttentionTensorContainer):
if not isinstance(args[1], AttentionTensorContainer) or not isinstance(args[2], AttentionTensorContainer):
raise TypeError("q, k, and v must all be attention tensor containers")
containers = args[:3]
remove_attn_wrapper_key = False
try:
if "_inside_attn_wrapper" not in kwargs:
transformer_options = kwargs.get("transformer_options", None)
remove_attn_wrapper_key = True
kwargs["_inside_attn_wrapper"] = True
if transformer_options is not None:
if "optimized_attention_override" in transformer_options:
optimized_attention_override = transformer_options["optimized_attention_override"]
if containers is not None:
if hasattr(optimized_attention_override, "container_function"):
return optimized_attention_override.container_function(*args, **kwargs)
args = tuple(container.take() for container in containers) + args[3:]
return optimized_attention_override(func, *args, **kwargs)
if containers is not None:
if wrapper.container_function is not None:
return wrapper.container_function(*args, **kwargs)
args = tuple(container.take() for container in containers) + args[3:]
return func(*args, **kwargs)
finally:
if remove_attn_wrapper_key:
del kwargs["_inside_attn_wrapper"]
wrapper.container_function = None
return wrapper
@wrap_attn
def attention_basic(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
attn_precision = get_attn_precision(attn_precision, q.dtype)
if skip_reshape:
b, _, _, dim_head = q.shape
else:
b, _, dim_head = q.shape
dim_head //= heads
scale = kwargs.get("scale", dim_head ** -0.5)
h = heads
if skip_reshape:
if kwargs.get("enable_gqa", False):
k, v = comfy.ops.repeat_kv_for_gqa(k, v, q.shape[-3], -3)
q, k, v = map(
lambda t: t.reshape(b * heads, -1, dim_head),
(q, k, v),
)
else:
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False))
q, k, v = map(lambda t: t.permute(0, 2, 1, 3).reshape(b * heads, -1, dim_head).contiguous(), (q, k, v))
# force cast to fp32 to avoid overflowing
if attn_precision == torch.float32:
sim = einsum('b i d, b j d -> b i j', q.float(), k.float()) * scale
else:
sim = einsum('b i d, b j d -> b i j', q, k) * scale
del q, k
if exists(mask):
if mask.dtype == torch.bool:
mask = rearrange(mask, 'b ... -> b (...)') #TODO: check if this bool part matches pytorch attention
max_neg_value = -torch.finfo(sim.dtype).max
mask = repeat(mask, 'b j -> (b h) () j', h=h)
sim.masked_fill_(~mask, max_neg_value)
else:
if len(mask.shape) == 2:
bs = 1
else:
bs = mask.shape[0]
mask = mask.reshape(bs, -1, mask.shape[-2], mask.shape[-1]).expand(b, heads, -1, -1).reshape(-1, mask.shape[-2], mask.shape[-1])
sim.add_(mask)
# attention, what we cannot get enough of
sim = sim.softmax(dim=-1)
out = einsum('b i j, b j d -> b i d', sim.to(v.dtype), v)
if skip_output_reshape:
out = (
out.unsqueeze(0)
.reshape(b, heads, -1, dim_head)
)
else:
out = (
out.unsqueeze(0)
.reshape(b, heads, -1, dim_head)
.permute(0, 2, 1, 3)
.reshape(b, -1, heads * dim_head)
)
return out
@wrap_attn
def attention_sub_quad(query, key, value, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
attn_precision = get_attn_precision(attn_precision, query.dtype)
if skip_reshape:
b, _, _, dim_head = query.shape
else:
b, _, dim_head = query.shape
dim_head //= heads
if "scale" in kwargs:
# Pre-scale query to match requested scale (cancels internal 1/sqrt(dim_head))
query = query * (kwargs["scale"] * dim_head ** 0.5)
if skip_reshape:
if kwargs.get("enable_gqa", False):
key, value = comfy.ops.repeat_kv_for_gqa(key, value, query.shape[-3], -3)
query = query.reshape(b * heads, -1, dim_head)
value = value.reshape(b * heads, -1, dim_head)
key = key.reshape(b * heads, -1, dim_head).movedim(1, 2)
else:
query, key, value = _reshape_qkv_to_heads(query, key, value, b, heads, dim_head, kwargs.get("enable_gqa", False))
query = query.permute(0, 2, 1, 3).reshape(b * heads, -1, dim_head)
value = value.permute(0, 2, 1, 3).reshape(b * heads, -1, dim_head)
key = key.permute(0, 2, 3, 1).reshape(b * heads, dim_head, -1)
dtype = query.dtype
upcast_attention = attn_precision == torch.float32 and query.dtype != torch.float32
if upcast_attention:
bytes_per_token = torch.finfo(torch.float32).bits//8
else:
bytes_per_token = torch.finfo(query.dtype).bits//8
batch_x_heads, q_tokens, _ = query.shape
_, _, k_tokens = key.shape
mem_free_total, _ = model_management.get_free_memory(query.device, True)
kv_chunk_size_min = None
kv_chunk_size = None
query_chunk_size = None
for x in [4096, 2048, 1024, 512, 256]:
count = mem_free_total / (batch_x_heads * bytes_per_token * x * 4.0)
if count >= k_tokens:
kv_chunk_size = k_tokens
query_chunk_size = x
break
if query_chunk_size is None:
query_chunk_size = 512
if mask is not None:
if len(mask.shape) == 2:
bs = 1
else:
bs = mask.shape[0]
mask = mask.reshape(bs, -1, mask.shape[-2], mask.shape[-1]).expand(b, heads, -1, -1).reshape(-1, mask.shape[-2], mask.shape[-1])
hidden_states = efficient_dot_product_attention(
query,
key,
value,
query_chunk_size=query_chunk_size,
kv_chunk_size=kv_chunk_size,
kv_chunk_size_min=kv_chunk_size_min,
use_checkpoint=False,
upcast_attention=upcast_attention,
mask=mask,
)
hidden_states = hidden_states.to(dtype)
if skip_output_reshape:
hidden_states = hidden_states.unflatten(0, (-1, heads))
else:
hidden_states = hidden_states.unflatten(0, (-1, heads)).transpose(1,2).flatten(start_dim=2)
return hidden_states
@wrap_attn
def attention_split(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
attn_precision = get_attn_precision(attn_precision, q.dtype)
if skip_reshape:
b, _, _, dim_head = q.shape
else:
b, _, dim_head = q.shape
dim_head //= heads
scale = kwargs.get("scale", dim_head ** -0.5)
if skip_reshape:
if kwargs.get("enable_gqa", False):
k, v = comfy.ops.repeat_kv_for_gqa(k, v, q.shape[-3], -3)
q, k, v = map(
lambda t: t.reshape(b * heads, -1, dim_head),
(q, k, v),
)
else:
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False))
q, k, v = map(lambda t: t.permute(0, 2, 1, 3).reshape(b * heads, -1, dim_head).contiguous(), (q, k, v))
r1 = torch.zeros(q.shape[0], q.shape[1], v.shape[2], device=q.device, dtype=q.dtype)
mem_free_total = model_management.get_free_memory(q.device)
if attn_precision == torch.float32:
element_size = 4
upcast = True
else:
element_size = q.element_size()
upcast = False
gb = 1024 ** 3
tensor_size = q.shape[0] * q.shape[1] * k.shape[1] * element_size
modifier = 3
mem_required = tensor_size * modifier
steps = 1
if mem_required > mem_free_total:
steps = 2**(math.ceil(math.log(mem_required / mem_free_total, 2)))
# print(f"Expected tensor size:{tensor_size/gb:0.1f}GB, cuda free:{mem_free_cuda/gb:0.1f}GB "
# f"torch free:{mem_free_torch/gb:0.1f} total:{mem_free_total/gb:0.1f} steps:{steps}")
if steps > 64:
max_res = math.floor(math.sqrt(math.sqrt(mem_free_total / 2.5)) / 8) * 64
raise RuntimeError(f'Not enough memory, use lower resolution (max approx. {max_res}x{max_res}). '
f'Need: {mem_required/64/gb:0.1f}GB free, Have:{mem_free_total/gb:0.1f}GB free')
if mask is not None:
if len(mask.shape) == 2:
bs = 1
else:
bs = mask.shape[0]
mask = mask.reshape(bs, -1, mask.shape[-2], mask.shape[-1]).expand(b, heads, -1, -1).reshape(-1, mask.shape[-2], mask.shape[-1])
# print("steps", steps, mem_required, mem_free_total, modifier, q.element_size(), tensor_size)
first_op_done = False
cleared_cache = False
while True:
try:
slice_size = q.shape[1] // steps if (q.shape[1] % steps) == 0 else q.shape[1]
for i in range(0, q.shape[1], slice_size):
end = i + slice_size
if upcast:
with torch.autocast(enabled=False, device_type = 'cuda'):
s1 = einsum('b i d, b j d -> b i j', q[:, i:end].float(), k.float()) * scale
else:
s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k) * scale
if mask is not None:
if len(mask.shape) == 2:
s1 += mask[i:end]
else:
if mask.shape[1] == 1:
s1 += mask
else:
s1 += mask[:, i:end]
s2 = s1.softmax(dim=-1).to(v.dtype)
del s1
first_op_done = True
r1[:, i:end] = einsum('b i j, b j d -> b i d', s2, v)
del s2
break
except Exception as e:
model_management.raise_non_oom(e)
if first_op_done == False:
model_management.soft_empty_cache(True)
if cleared_cache == False:
cleared_cache = True
logging.warning("out of memory error, emptying cache and trying again")
continue
steps *= 2
if steps > 64:
raise e
logging.warning("out of memory error, increasing steps and trying again {}".format(steps))
else:
raise e
del q, k, v
if skip_output_reshape:
r1 = (
r1.unsqueeze(0)
.reshape(b, heads, -1, dim_head)
)
else:
r1 = (
r1.unsqueeze(0)
.reshape(b, heads, -1, dim_head)
.permute(0, 2, 1, 3)
.reshape(b, -1, heads * dim_head)
)
return r1
BROKEN_XFORMERS = False
try:
x_vers = xformers.__version__
# XFormers bug confirmed on all versions from 0.0.21 to 0.0.26 (q with bs bigger than 65535 gives CUDA error)
BROKEN_XFORMERS = x_vers.startswith("0.0.2") and not x_vers.startswith("0.0.20")
except:
pass
@wrap_attn
def attention_xformers(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
b = q.shape[0]
dim_head = q.shape[-1]
# check to make sure xformers isn't broken
disabled_xformers = False
if BROKEN_XFORMERS:
if b * heads > 65535:
disabled_xformers = True
if not disabled_xformers:
if torch.jit.is_tracing() or torch.jit.is_scripting():
disabled_xformers = True
if disabled_xformers:
return attention_pytorch(q, k, v, heads, mask, skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs)
if skip_reshape:
# b h k d -> b k h d
q, k, v = map(
lambda t: t.permute(0, 2, 1, 3),
(q, k, v),
)
if kwargs.get("enable_gqa", False):
k, v = comfy.ops.repeat_kv_for_gqa(k, v, q.shape[-2], -2)
# actually do the reshaping
else:
dim_head //= heads
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False))
if mask is not None:
# add a singleton batch dimension
if mask.ndim == 2:
mask = mask.unsqueeze(0)
# add a singleton heads dimension
if mask.ndim == 3:
mask = mask.unsqueeze(1)
# pad to a multiple of 8
pad = 8 - mask.shape[-1] % 8
# the xformers docs says that it's allowed to have a mask of shape (1, Nq, Nk)
# but when using separated heads, the shape has to be (B, H, Nq, Nk)
# in flux, this matrix ends up being over 1GB
# here, we create a mask with the same batch/head size as the input mask (potentially singleton or full)
mask_out = torch.empty([mask.shape[0], mask.shape[1], q.shape[1], mask.shape[-1] + pad], dtype=q.dtype, device=q.device)
mask_out[..., :mask.shape[-1]] = mask
# doesn't this remove the padding again??
mask = mask_out[..., :mask.shape[-1]]
mask = mask.expand(b, heads, -1, -1)
out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=mask, scale=kwargs.get("scale", None))
if skip_output_reshape:
out = out.permute(0, 2, 1, 3)
else:
out = (
out.reshape(b, -1, heads * dim_head)
)
return out
if model_management.is_nvidia(): #pytorch 2.3 and up seem to have this issue.
SDP_BATCH_LIMIT = 2**15
else:
#TODO: other GPUs ?
SDP_BATCH_LIMIT = 2**31
@wrap_attn
def attention_pytorch(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
if skip_reshape:
b, _, _, dim_head = q.shape
else:
b, _, dim_head = q.shape
dim_head //= heads
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False), expand_kv=False)
q, k, v = map(lambda t: t.transpose(1, 2), (q, k, v))
if mask is not None:
# add a batch dimension if there isn't already one
if mask.ndim == 2:
mask = mask.unsqueeze(0)
# add a heads dimension if there isn't already one
if mask.ndim == 3:
mask = mask.unsqueeze(1)
sdpa_keys = ("scale", "enable_gqa")
sdpa_extra = {k: v for k, v in kwargs.items() if k in sdpa_keys}
if SDP_BATCH_LIMIT >= b:
out = comfy.ops.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False, **sdpa_extra)
if not skip_output_reshape:
out = (
out.transpose(1, 2).reshape(b, -1, heads * dim_head)
)
else:
out = torch.empty((b, q.shape[2], heads * dim_head), dtype=q.dtype, layout=q.layout, device=q.device)
for i in range(0, b, SDP_BATCH_LIMIT):
m = mask
if mask is not None:
if mask.shape[0] > 1:
m = mask[i : i + SDP_BATCH_LIMIT]
out[i : i + SDP_BATCH_LIMIT] = comfy.ops.scaled_dot_product_attention(
q[i : i + SDP_BATCH_LIMIT],
k[i : i + SDP_BATCH_LIMIT],
v[i : i + SDP_BATCH_LIMIT],
attn_mask=m,
dropout_p=0.0, is_causal=False, **sdpa_extra
).transpose(1, 2).reshape(-1, q.shape[2], heads * dim_head)
return out
def _comfy_kitchen_int8_inputs(q, k, v, heads, mask, skip_reshape, enable_gqa):
dim_head = q.shape[-1] if skip_reshape else q.shape[-1] // heads
b = q.shape[0]
if not skip_reshape:
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, enable_gqa, expand_kv=False)
q, k, v = map(lambda t: t.transpose(1, 2), (q, k, v))
if mask is not None:
if mask.ndim == 2:
mask = mask.unsqueeze(0)
if mask.ndim == 3:
mask = mask.unsqueeze(1)
return q, k, v, mask, b, dim_head
@wrap_attn
def attention_comfy_kitchen_int8(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
q, k, v, mask, b, dim_head = _comfy_kitchen_int8_inputs(
q, k, v, heads, mask, skip_reshape, kwargs.get("enable_gqa", False)
)
out = comfy_kitchen.int8_attention(
q,
k,
v,
scale=kwargs.get("scale", None),
attn_mask=mask,
)
if not skip_output_reshape:
out = out.transpose(1, 2).reshape(b, -1, heads * dim_head)
return out
def _attention_comfy_kitchen_int8_containers(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
q = q.take()
k = k.take()
v = v.take()
q, k, v, mask, b, dim_head = _comfy_kitchen_int8_inputs(
q, k, v, heads, mask, skip_reshape, kwargs.get("enable_gqa", False)
)
quantized = comfy_kitchen.prequantize_int8_attention(
q,
k,
v,
scale=kwargs.get("scale", None),
attn_mask=mask,
)
del q, k, v
out = comfy_kitchen.int8_attention_from_prequantized(quantized)
if not skip_output_reshape:
out = out.transpose(1, 2).reshape(b, -1, heads * dim_head)
return out
attention_comfy_kitchen_int8.container_function = _attention_comfy_kitchen_int8_containers
@wrap_attn
def attention_sage(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
if kwargs.get("low_precision_attention", True) is False or (mask is not None and not SAGE_ATTENTION_SUPPORTS_MASK):
return attention_pytorch(q, k, v, heads, mask=mask, skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs)
exception_fallback = False
if skip_reshape:
b, _, _, dim_head = q.shape
tensor_layout = "HND"
if kwargs.get("enable_gqa", False):
k, v = comfy.ops.repeat_kv_for_gqa(k, v, q.shape[-3], -3)
else:
b, _, dim_head = q.shape
dim_head //= heads
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False))
tensor_layout = "NHD"
if mask is not None:
# add a batch dimension if there isn't already one
if mask.ndim == 2:
mask = mask.unsqueeze(0)
# add a heads dimension if there isn't already one
if mask.ndim == 3:
mask = mask.unsqueeze(1)
sage_kwargs = {"is_causal": False, "tensor_layout": tensor_layout, "sm_scale": kwargs.get("scale", None), "smooth_k": False}
if mask is not None:
sage_kwargs["attn_mask"] = mask
try:
out = sageattn(q, k, v, **sage_kwargs)
except Exception as e:
logging.error("Error running sage attention: {}, using pytorch attention instead.".format(e))
exception_fallback = True
if exception_fallback:
if tensor_layout == "NHD":
q, k, v = map(
lambda t: t.transpose(1, 2),
(q, k, v),
)
return attention_pytorch(q, k, v, heads, mask=mask, skip_reshape=True, skip_output_reshape=skip_output_reshape, **kwargs)
if tensor_layout == "HND":
if not skip_output_reshape:
out = (
out.transpose(1, 2).reshape(b, -1, heads * dim_head)
)
else:
if skip_output_reshape:
out = out.transpose(1, 2)
else:
out = out.reshape(b, -1, heads * dim_head)
return out
@wrap_attn
def attention3_sage(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
exception_fallback = False
if (q.device.type != "cuda" or
q.dtype not in (torch.float16, torch.bfloat16) or
mask is not None):
return attention_pytorch(
q, k, v, heads,
mask=mask,
attn_precision=attn_precision,
skip_reshape=skip_reshape,
skip_output_reshape=skip_output_reshape,
**kwargs
)
if skip_reshape:
B, H, L, D = q.shape
if H != heads:
return attention_pytorch(
q, k, v, heads,
mask=mask,
attn_precision=attn_precision,
skip_reshape=True,
skip_output_reshape=skip_output_reshape,
**kwargs
)
N = q.shape[2]
dim_head = D
else:
B, N, inner_dim = q.shape
if inner_dim % heads != 0:
return attention_pytorch(
q, k, v, heads,
mask=mask,
attn_precision=attn_precision,
skip_reshape=False,
skip_output_reshape=skip_output_reshape,
**kwargs
)
dim_head = inner_dim // heads
if dim_head >= 256 or N <= 1024:
return attention_pytorch(
q, k, v, heads,
mask=mask,
attn_precision=attn_precision,
skip_reshape=skip_reshape,
skip_output_reshape=skip_output_reshape,
**kwargs
)
if skip_reshape:
q_s = q
if kwargs.get("enable_gqa", False):
k_s, v_s = comfy.ops.repeat_kv_for_gqa(k, v, H, -3)
else:
k_s, v_s = k, v
else:
q_s, k_s, v_s = _reshape_qkv_to_heads(q, k, v, B, heads, dim_head, kwargs.get("enable_gqa", False))
q_s, k_s, v_s = map(lambda t: t.permute(0, 2, 1, 3).contiguous(), (q_s, k_s, v_s))
B, H, L, D = q_s.shape
try:
out = sageattn3_blackwell(q_s, k_s, v_s, is_causal=False)
except Exception as e:
exception_fallback = True
logging.error("Error running SageAttention3: %s, falling back to pytorch attention.", e)
if exception_fallback:
if not skip_reshape:
del q_s, k_s, v_s
return attention_pytorch(
q, k, v, heads,
mask=mask,
attn_precision=attn_precision,
skip_reshape=skip_reshape,
skip_output_reshape=skip_output_reshape,
**kwargs
)
if skip_reshape:
if not skip_output_reshape:
out = out.permute(0, 2, 1, 3).reshape(B, L, H * D)
else:
if skip_output_reshape:
pass
else:
out = out.permute(0, 2, 1, 3).reshape(B, L, H * D)
return out
try:
@torch.library.custom_op("comfy::flash_attn", mutates_args=())
def flash_attn_wrapper(q: torch.Tensor, k: torch.Tensor, v: torch.Tensor,
dropout_p: float = 0.0, causal: bool = False, softmax_scale: float = -1.0) -> torch.Tensor:
softmax_scale_arg = None if softmax_scale == -1.0 else softmax_scale
return flash_attn_func(q, k, v, dropout_p=dropout_p, causal=causal, softmax_scale=softmax_scale_arg)
@flash_attn_wrapper.register_fake
def flash_attn_fake(q, k, v, dropout_p=0.0, causal=False, softmax_scale=-1.0):
# Output shape is the same as q
return q.new_empty(q.shape)
except AttributeError as error:
FLASH_ATTN_ERROR = error
def flash_attn_wrapper(q: torch.Tensor, k: torch.Tensor, v: torch.Tensor,
dropout_p: float = 0.0, causal: bool = False, softmax_scale: float = -1.0) -> torch.Tensor:
assert False, f"Could not define flash_attn_wrapper: {FLASH_ATTN_ERROR}"
@wrap_attn
def attention_flash(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):
if skip_reshape:
b, _, _, dim_head = q.shape
else:
b, _, dim_head = q.shape
dim_head //= heads
q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head, kwargs.get("enable_gqa", False), expand_kv=False)
q, k, v = map(lambda t: t.transpose(1, 2), (q, k, v))
if mask is not None:
# add a batch dimension if there isn't already one
if mask.ndim == 2:
mask = mask.unsqueeze(0)
# add a heads dimension if there isn't already one
if mask.ndim == 3:
mask = mask.unsqueeze(1)
try:
if mask is not None:
raise RuntimeError("Mask must not be set for Flash attention")
if FLASH_ATTENTION_BACKEND == "triton":
# The Triton implementation expects [batch, heads, sequence, head_dim].
out = triton_flash_attn_func(
q,
k,
v,
dropout_p=0.0,
causal=False,
softmax_scale=kwargs.get("scale"),
)
else:
# The native flash-attn implementation expects
# [batch, sequence, heads, head_dim].
native_q, native_k, native_v = q, k, v
if FLASH_ATTN_REPACK_QKV:
# q/k/v are [batch, heads, sequence, head_dim]. ComfyUI usually
# reaches this point as a view over contiguous BNHD storage,
# which makes adjacent tokens for one head H*D elements apart.
# Repacking BHND makes each head's token rows adjacent before
# exposing the BNHD view expected by native flash-attn.
native_q = native_q.contiguous()
native_k = native_k.contiguous()
native_v = native_v.contiguous()
out = flash_attn_wrapper(
native_q.transpose(1, 2),
native_k.transpose(1, 2),
native_v.transpose(1, 2),
dropout_p=0.0,
causal=False,
softmax_scale=kwargs.get("scale", -1.0),
).transpose(1, 2)
except Exception as e:
logging.warning(f"{FLASH_ATTENTION_BACKEND} Flash Attention failed, using default SDPA: {e}")
sdpa_extra = {}
if kwargs.get("enable_gqa", False):
sdpa_extra["enable_gqa"] = True
if "scale" in kwargs:
sdpa_extra["scale"] = kwargs["scale"]
out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False, **sdpa_extra)
if not skip_output_reshape:
out = (
out.transpose(1, 2).reshape(b, -1, heads * dim_head)
)
return out
optimized_attention = attention_basic
if model_management.sage_attention_enabled():
logging.info("Using sage attention")
optimized_attention = attention_sage
elif model_management.flash_attention_enabled():
logging.info("Using Flash Attention")
optimized_attention = attention_flash
elif model_management.xformers_enabled():
logging.info("Using xformers attention")
optimized_attention = attention_xformers
elif model_management.pytorch_attention_enabled():
logging.info("Using pytorch attention")
optimized_attention = attention_pytorch
else:
if args.use_split_cross_attention:
logging.info("Using split optimization for attention")
optimized_attention = attention_split
else:
logging.info("Using sub quadratic optimization for attention, if you have memory or speed issues try using: --use-split-cross-attention")
optimized_attention = attention_sub_quad
if model_management.comfy_kitchen_attention_enabled():
if COMFY_KITCHEN_INT8_ATTENTION_IS_AVAILABLE:
logging.info("Using Comfy Kitchen attention")
optimized_attention = attention_comfy_kitchen_int8
else:
logging.error("Comfy Kitchen attention is unavailable. Install a Comfy Kitchen build with attention support to use --use-ck-attention.")
exit(-1)
optimized_attention_masked = optimized_attention
# register core-supported attention functions
if COMFY_KITCHEN_INT8_ATTENTION_IS_AVAILABLE:
register_attention_function("comfy_kitchen_int8", attention_comfy_kitchen_int8)
if SAGE_ATTENTION_IS_AVAILABLE:
register_attention_function("sage", attention_sage)
if SAGE_ATTENTION3_IS_AVAILABLE:
register_attention_function("sage3", attention3_sage)
if FLASH_ATTENTION_IS_AVAILABLE:
register_attention_function("flash", attention_flash)
if model_management.xformers_enabled():
register_attention_function("xformers", attention_xformers)
register_attention_function("pytorch", attention_pytorch)
register_attention_function("sub_quad", attention_sub_quad)
register_attention_function("split", attention_split)
def optimized_attention_for_device(device, mask=False, small_input=False):
if small_input:
if model_management.pytorch_attention_enabled():
return attention_pytorch #TODO: need to confirm but this is probably slightly faster for small inputs in all cases
else:
return attention_basic
if device == torch.device("cpu"):
return attention_sub_quad
if mask:
return optimized_attention_masked
return optimized_attention
class CrossAttention(nn.Module):
def __init__(self, query_dim, context_dim=None, heads=8, dim_head=64, dropout=0., attn_precision=None, dtype=None, device=None, operations=ops):
super().__init__()
inner_dim = dim_head * heads
context_dim = default(context_dim, query_dim)
self.attn_precision = attn_precision
self.heads = heads
self.dim_head = dim_head
self.to_q = operations.Linear(query_dim, inner_dim, bias=False, dtype=dtype, device=device)
self.to_k = operations.Linear(context_dim, inner_dim, bias=False, dtype=dtype, device=device)
self.to_v = operations.Linear(context_dim, inner_dim, bias=False, dtype=dtype, device=device)
self.to_out = nn.Sequential(operations.Linear(inner_dim, query_dim, dtype=dtype, device=device), nn.Dropout(dropout))
def forward(self, x, context=None, value=None, mask=None, transformer_options={}):
q = self.to_q(x)
context = default(context, x)
k = self.to_k(context)
if value is not None:
v = self.to_v(value)
del value
else:
v = self.to_v(context)
if mask is None:
out = optimized_attention(q, k, v, self.heads, attn_precision=self.attn_precision, transformer_options=transformer_options)
else:
out = optimized_attention_masked(q, k, v, self.heads, mask, attn_precision=self.attn_precision, transformer_options=transformer_options)
return self.to_out(out)
class BasicTransformerBlock(nn.Module):
def __init__(self, dim, n_heads, d_head, dropout=0., context_dim=None, gated_ff=True, checkpoint=True, ff_in=False, inner_dim=None,
disable_self_attn=False, disable_temporal_crossattention=False, switch_temporal_ca_to_sa=False, attn_precision=None, dtype=None, device=None, operations=ops):
super().__init__()
self.ff_in = ff_in or inner_dim is not None
if inner_dim is None:
inner_dim = dim
self.is_res = inner_dim == dim
self.attn_precision = attn_precision
if self.ff_in:
self.norm_in = operations.LayerNorm(dim, dtype=dtype, device=device)
self.ff_in = FeedForward(dim, dim_out=inner_dim, dropout=dropout, glu=gated_ff, dtype=dtype, device=device, operations=operations)
self.disable_self_attn = disable_self_attn
self.attn1 = CrossAttention(query_dim=inner_dim, heads=n_heads, dim_head=d_head, dropout=dropout,
context_dim=context_dim if self.disable_self_attn else None, attn_precision=self.attn_precision, dtype=dtype, device=device, operations=operations) # is a self-attention if not self.disable_self_attn
self.ff = FeedForward(inner_dim, dim_out=dim, dropout=dropout, glu=gated_ff, dtype=dtype, device=device, operations=operations)
if disable_temporal_crossattention:
if switch_temporal_ca_to_sa:
raise ValueError
else:
self.attn2 = None
else:
context_dim_attn2 = None
if not switch_temporal_ca_to_sa:
context_dim_attn2 = context_dim
self.attn2 = CrossAttention(query_dim=inner_dim, context_dim=context_dim_attn2,
heads=n_heads, dim_head=d_head, dropout=dropout, attn_precision=self.attn_precision, dtype=dtype, device=device, operations=operations) # is self-attn if context is none
self.norm2 = operations.LayerNorm(inner_dim, dtype=dtype, device=device)
self.norm1 = operations.LayerNorm(inner_dim, dtype=dtype, device=device)
self.norm3 = operations.LayerNorm(inner_dim, dtype=dtype, device=device)
self.n_heads = n_heads
self.d_head = d_head
self.switch_temporal_ca_to_sa = switch_temporal_ca_to_sa
def forward(self, x, context=None, transformer_options={}):
extra_options = {}
block = transformer_options.get("block", None)
block_index = transformer_options.get("block_index", 0)
transformer_patches = {}
transformer_patches_replace = {}
for k in transformer_options:
if k == "patches":
transformer_patches = transformer_options[k]
elif k == "patches_replace":
transformer_patches_replace = transformer_options[k]
else:
extra_options[k] = transformer_options[k]
extra_options["n_heads"] = self.n_heads
extra_options["dim_head"] = self.d_head
extra_options["attn_precision"] = self.attn_precision
if self.ff_in:
x_skip = x
x = self.ff_in(self.norm_in(x))
if self.is_res:
x += x_skip
n = self.norm1(x)
if self.disable_self_attn:
context_attn1 = context
else:
context_attn1 = None
value_attn1 = None
if "attn1_patch" in transformer_patches:
patch = transformer_patches["attn1_patch"]
if context_attn1 is None:
context_attn1 = n
value_attn1 = context_attn1
for p in patch:
n, context_attn1, value_attn1 = p(n, context_attn1, value_attn1, extra_options)
if block is not None:
transformer_block = (block[0], block[1], block_index)
else:
transformer_block = None
attn1_replace_patch = transformer_patches_replace.get("attn1", {})
block_attn1 = transformer_block
if block_attn1 not in attn1_replace_patch:
block_attn1 = block
if block_attn1 in attn1_replace_patch:
if context_attn1 is None:
context_attn1 = n
value_attn1 = n
n = self.attn1.to_q(n)
context_attn1 = self.attn1.to_k(context_attn1)
value_attn1 = self.attn1.to_v(value_attn1)
n = attn1_replace_patch[block_attn1](n, context_attn1, value_attn1, extra_options)
n = self.attn1.to_out(n)
else:
n = self.attn1(n, context=context_attn1, value=value_attn1, transformer_options=transformer_options)
if "attn1_output_patch" in transformer_patches:
patch = transformer_patches["attn1_output_patch"]
for p in patch:
n = p(n, extra_options)
x = n + x
if "middle_patch" in transformer_patches:
patch = transformer_patches["middle_patch"]
for p in patch:
x = p(x, extra_options)
if self.attn2 is not None:
n = self.norm2(x)
if self.switch_temporal_ca_to_sa:
context_attn2 = n
else:
context_attn2 = context
value_attn2 = None
if "attn2_patch" in transformer_patches:
patch = transformer_patches["attn2_patch"]
value_attn2 = context_attn2
for p in patch:
n, context_attn2, value_attn2 = p(n, context_attn2, value_attn2, extra_options)
attn2_replace_patch = transformer_patches_replace.get("attn2", {})
block_attn2 = transformer_block
if block_attn2 not in attn2_replace_patch:
block_attn2 = block
if block_attn2 in attn2_replace_patch:
if value_attn2 is None:
value_attn2 = context_attn2
n = self.attn2.to_q(n)
context_attn2 = self.attn2.to_k(context_attn2)
value_attn2 = self.attn2.to_v(value_attn2)
n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options)
n = self.attn2.to_out(n)
else:
n = self.attn2(n, context=context_attn2, value=value_attn2, transformer_options=transformer_options)
if "attn2_output_patch" in transformer_patches:
patch = transformer_patches["attn2_output_patch"]
for p in patch:
n = p(n, extra_options)
x = n + x
if self.is_res:
x_skip = x
x = self.ff(self.norm3(x))
if self.is_res:
x = x_skip + x
return x
class SpatialTransformer(nn.Module):
"""
Transformer block for image-like data.
First, project the input (aka embedding)
and reshape to b, t, d.
Then apply standard transformer action.
Finally, reshape to image
NEW: use_linear for more efficiency instead of the 1x1 convs
"""
def __init__(self, in_channels, n_heads, d_head,
depth=1, dropout=0., context_dim=None,
disable_self_attn=False, use_linear=False,
use_checkpoint=True, attn_precision=None, dtype=None, device=None, operations=ops):
super().__init__()
if exists(context_dim) and not isinstance(context_dim, list):
context_dim = [context_dim] * depth
self.in_channels = in_channels
inner_dim = n_heads * d_head
self.norm = operations.GroupNorm(num_groups=32, num_channels=in_channels, eps=1e-6, affine=True, dtype=dtype, device=device)
if not use_linear:
self.proj_in = operations.Conv2d(in_channels,
inner_dim,
kernel_size=1,
stride=1,
padding=0, dtype=dtype, device=device)
else:
self.proj_in = operations.Linear(in_channels, inner_dim, dtype=dtype, device=device)
self.transformer_blocks = nn.ModuleList(
[BasicTransformerBlock(inner_dim, n_heads, d_head, dropout=dropout, context_dim=context_dim[d],
disable_self_attn=disable_self_attn, checkpoint=use_checkpoint, attn_precision=attn_precision, dtype=dtype, device=device, operations=operations)
for d in range(depth)]
)
if not use_linear:
self.proj_out = operations.Conv2d(inner_dim,in_channels,
kernel_size=1,
stride=1,
padding=0, dtype=dtype, device=device)
else:
self.proj_out = operations.Linear(in_channels, inner_dim, dtype=dtype, device=device)
self.use_linear = use_linear
def forward(self, x, context=None, transformer_options={}):
# note: if no context is given, cross-attention defaults to self-attention
if not isinstance(context, list):
context = [context] * len(self.transformer_blocks)
b, c, h, w = x.shape
transformer_options["activations_shape"] = list(x.shape)
x_in = x
x = self.norm(x)
if not self.use_linear:
x = self.proj_in(x)
x = x.movedim(1, 3).flatten(1, 2).contiguous()
if self.use_linear:
x = self.proj_in(x)
for i, block in enumerate(self.transformer_blocks):
transformer_options["block_index"] = i
x = block(x, context=context[i], transformer_options=transformer_options)
if self.use_linear:
x = self.proj_out(x)
x = x.reshape(x.shape[0], h, w, x.shape[-1]).movedim(3, 1).contiguous()
if not self.use_linear:
x = self.proj_out(x)
return x + x_in
class SpatialVideoTransformer(SpatialTransformer):
def __init__(
self,
in_channels,
n_heads,
d_head,
depth=1,
dropout=0.0,
use_linear=False,
context_dim=None,
use_spatial_context=False,
timesteps=None,
merge_strategy: str = "fixed",
merge_factor: float = 0.5,
time_context_dim=None,
ff_in=False,
checkpoint=False,
time_depth=1,
disable_self_attn=False,
disable_temporal_crossattention=False,
max_time_embed_period: int = 10000,
attn_precision=None,
dtype=None, device=None, operations=ops
):
super().__init__(
in_channels,
n_heads,
d_head,
depth=depth,
dropout=dropout,
use_checkpoint=checkpoint,
context_dim=context_dim,
use_linear=use_linear,
disable_self_attn=disable_self_attn,
attn_precision=attn_precision,
dtype=dtype, device=device, operations=operations
)
self.time_depth = time_depth
self.depth = depth
self.max_time_embed_period = max_time_embed_period
time_mix_d_head = d_head
n_time_mix_heads = n_heads
time_mix_inner_dim = int(time_mix_d_head * n_time_mix_heads)
inner_dim = n_heads * d_head
if use_spatial_context:
time_context_dim = context_dim
self.time_stack = nn.ModuleList(
[
BasicTransformerBlock(
inner_dim,
n_time_mix_heads,
time_mix_d_head,
dropout=dropout,
context_dim=time_context_dim,
# timesteps=timesteps,
checkpoint=checkpoint,
ff_in=ff_in,
inner_dim=time_mix_inner_dim,
disable_self_attn=disable_self_attn,
disable_temporal_crossattention=disable_temporal_crossattention,
attn_precision=attn_precision,
dtype=dtype, device=device, operations=operations
)
for _ in range(self.depth)
]
)
assert len(self.time_stack) == len(self.transformer_blocks)
self.use_spatial_context = use_spatial_context
self.in_channels = in_channels
time_embed_dim = self.in_channels * 4
self.time_pos_embed = nn.Sequential(
operations.Linear(self.in_channels, time_embed_dim, dtype=dtype, device=device),
nn.SiLU(),
operations.Linear(time_embed_dim, self.in_channels, dtype=dtype, device=device),
)
self.time_mixer = AlphaBlender(
alpha=merge_factor, merge_strategy=merge_strategy
)
def forward(
self,
x: torch.Tensor,
context: Optional[torch.Tensor] = None,
time_context: Optional[torch.Tensor] = None,
timesteps: Optional[int] = None,
image_only_indicator: Optional[torch.Tensor] = None,
transformer_options={}
) -> torch.Tensor:
_, _, h, w = x.shape
transformer_options["activations_shape"] = list(x.shape)
x_in = x
spatial_context = None
if exists(context):
spatial_context = context
if self.use_spatial_context:
assert (
context.ndim == 3
), f"n dims of spatial context should be 3 but are {context.ndim}"
if time_context is None:
time_context = context
time_context_first_timestep = time_context[::timesteps]
time_context = repeat(
time_context_first_timestep, "b ... -> (b n) ...", n=h * w
)
elif time_context is not None and not self.use_spatial_context:
time_context = repeat(time_context, "b ... -> (b n) ...", n=h * w)
if time_context.ndim == 2:
time_context = rearrange(time_context, "b c -> b 1 c")
x = self.norm(x)
if not self.use_linear:
x = self.proj_in(x)
x = rearrange(x, "b c h w -> b (h w) c")
if self.use_linear:
x = self.proj_in(x)
num_frames = torch.arange(timesteps, device=x.device)
num_frames = repeat(num_frames, "t -> b t", b=x.shape[0] // timesteps)
num_frames = rearrange(num_frames, "b t -> (b t)")
t_emb = timestep_embedding(num_frames, self.in_channels, repeat_only=False, max_period=self.max_time_embed_period).to(x.dtype)
emb = self.time_pos_embed(t_emb)
emb = emb[:, None, :]
for it_, (block, mix_block) in enumerate(
zip(self.transformer_blocks, self.time_stack)
):
transformer_options["block_index"] = it_
x = block(
x,
context=spatial_context,
transformer_options=transformer_options,
)
x_mix = x
x_mix = x_mix + emb
B, S, C = x_mix.shape
x_mix = rearrange(x_mix, "(b t) s c -> (b s) t c", t=timesteps)
x_mix = mix_block(x_mix, context=time_context, transformer_options=transformer_options)
x_mix = rearrange(
x_mix, "(b s) t c -> (b t) s c", s=S, b=B // timesteps, c=C, t=timesteps
)
x = self.time_mixer(x_spatial=x, x_temporal=x_mix, image_only_indicator=image_only_indicator)
if self.use_linear:
x = self.proj_out(x)
x = rearrange(x, "b (h w) c -> b c h w", h=h, w=w)
if not self.use_linear:
x = self.proj_out(x)
out = x + x_in
return out这个修改承担了三层核心使命，直接决定了你文章里"7分钟生成15秒视频"的结果能否稳定复现：
1. 硬件级"避雷"：强制规避K100_AI的VM Fault（首要生存目的）
这是修改最根本、不可妥协 的目的。K100_AI的共享内存（LDS）只有64KB，而flash-attn默认的Triton内核会自动尝试BLOCK_M=256的配置，这会直接撑爆LDS并触发页错误（VM Fault），导致内核崩溃或静默生成花屏视频。
-
修改手段 ：代码强制将Triton内核的
BLOCK_M和BLOCK_N锁定为64，并将num_warps降为4。 -
最终目的 ：用"降频"换取"生存"，确保在长达15秒的视频生成长序列（5万+ tokens）推理中，不会因为硬件资源超限而中途宕机。
2. 软件栈"缝合"：修复DTK Triton与Flash-Attn的版本断裂
海光DTK 2604内置的Triton是3.6.x版本，而flash-attn 2.8.x依赖的get_best_config API在Triton 3.6中已被删除。若不干预，ComfyUI启动时就会报AttributeError直接退出。
-
修改手段 ：利用Python动态特性，在运行时手动补齐 了缺失的
get_best_config方法（猴子补丁）。 -
最终目的 ：在不允许升级Triton（升级会导致与DTK驱动不兼容）的前提下，让新版flash-attn强行跑在旧版Triton上，打通了官方生态与国产软件栈之间的"断头路"。
3. 显存访问"调优"：消除非连续内存带来的隐形损耗
ComfyUI原生张量布局是BNHD（Batch-Heads-Seq），而原生Flash Attention要求BSHD（Batch-Seq-Heads）。直接做transpose会产生非连续内存视图，导致Flash Attention内部进行低效的隐式拷贝，拖慢推理速度。
-
修改手段 ：在
transpose之前强制调用contiguous()，并配合启动参数COMFY_FLASH_ATTN_REPACK_QKV=1，主动将QKV重排为连续内存。 -
最终目的 ：虽然多了一次显式拷贝，但消除了内核内部的寻址跳转开销。实测在你的K100_AI上，这个操作反而让端到端速度提升了5%~7%，真正榨干了显存带宽（896GB/s）。
六、启动脚本与多实例并行
6.1 单卡单实例启动脚本
bash
#!/bin/bash
export HIP_VISIBLE_DEVICES=7
BASE_PORT=7681
BASE_DB_PATH="/home/models/ComfyUI-master/user/comfyui"
PYTORCH_ALLOC_CONF=expandable_segments:True \
COMFY_FLASH_ATTN_BACKEND=native \
COMFY_FLASH_ATTN_REPACK_QKV=1 \
python main.py \
--use-flash-attention \
--listen 0.0.0.0 \
--port $BASE_PORT \
--database-url "sqlite:///${BASE_DB_PATH}.db"参数解读：
-
HIP_VISIBLE_DEVICES=7：指定使用第7号DCU（0-7共8卡） -
PYTORCH_ALLOC_CONF=expandable_segments:True：允许PyTorch扩展显存段，减少碎片 -
COMFY_FLASH_ATTN_BACKEND=native：使用原生Flash Attention后端 -
COMFY_FLASH_ATTN_REPACK_QKV=1：启用QKV重排优化 -
--use-flash-attention：启用Flash Attention加速
6.2 多实例并行策略
由于是单卡运行，可以开启8个ComfyUI实例 同时运行（每卡一个实例）。通过修改HIP_VISIBLE_DEVICES（0-7）和BASE_PORT（7681-7688），每个实例绑定不同的DCU和端口。
并行收益：
-
单卡15秒高清视频：约22分钟
-
8卡并行：约20分钟完成8个视频
-
总产出：20分钟生成120秒768P高清视频
6.3 测试工作流
bash
{
"id": "e3f2b845-8f2c-4b5a-9caf-eac1029d3e7e",
"revision": 0,
"last_node_id": 122,
"last_link_id": 238,
"nodes": [
{
"id": 92,
"type": "SaveVideo",
"pos": [
-532.2764838467319,
4771.015641008128
],
"size": [
1070,
358
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "video",
"type": "VIDEO",
"link": 194
}
],
"outputs": [
{
"name": "video",
"type": "VIDEO",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "SaveVideo"
},
"widgets_values": [
"video/MiniMax_H3",
"auto",
"auto",
"auto"
],
"widgets_values_named": {
"filename_prefix": "video/MiniMax_H3",
"format": "auto",
"format.codec": "auto",
"codec": "auto"
}
},
{
"id": 115,
"type": "ResolutionSelector",
"pos": [
-1487.5,
4780.227272727273
],
"size": [
270,
170
],
"flags": {},
"order": 0,
"mode": 0,
"showAdvanced": true,
"inputs": [],
"outputs": [
{
"name": "width",
"type": "INT",
"links": [
219
]
},
{
"name": "height",
"type": "INT",
"links": [
220
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "ResolutionSelector"
},
"widgets_values": [
"9:16 (Portrait Widescreen)",
1,
32
],
"widgets_values_named": {
"aspect_ratio": "9:16 (Portrait Widescreen)",
"megapixels": 1,
"multiple": 32
}
},
{
"id": 105,
"type": "4c314f31-ecda-4b08-ae98-faaba1bf613f",
"pos": [
-1125.662674832472,
4474.038699956329
],
"size": [
480,
630
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "first_frame",
"shape": 7,
"type": "IMAGE",
"link": null
},
{
"name": "last_frame",
"shape": 7,
"type": "IMAGE",
"link": null
},
{
"name": "width",
"type": "INT",
"widget": {
"name": "width"
},
"link": 219
},
{
"name": "height",
"type": "INT",
"widget": {
"name": "height"
},
"link": 220
},
{
"label": "duration",
"name": "value_1",
"type": "FLOAT",
"widget": {
"name": "value_1"
},
"link": null
},
{
"label": "audio_vae",
"name": "vae_name_1",
"type": "COMBO",
"widget": {
"name": "vae_name_1"
},
"link": null
}
],
"outputs": [
{
"name": "VIDEO",
"type": "VIDEO",
"links": [
194
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"previewExposures": []
},
"widgets_values": [
"[0s-3s] Medium wide shot of a modern Chinese cybersecurity command center at night. Two young Chinese cyber police officers --- a male and a female, both in sharp dark blue police uniforms with badges and epaulets --- stand before a massive curved LED wall displaying scrolling code, network topology maps, and real-time data streams. The room is dimly lit with cool blue ambient light from the screens, casting soft reflections on their faces. The male officer points at a suspicious data spike on the screen, exchanging a serious glance with his female partner. The screens flicker with red warning indicators. Audio: subtle electronic beeps, keyboard clicks, and low-frequency digital hum. Camera slowly pushes in toward the officers from a slight low angle, building tension.\n\n[3s-6s] A dramatic whip pan transitions to a split-screen composition. On the left side, the two police officers are now seated at a high-tech console, hands moving rapidly across holographic keyboards, eyes locked on floating transparent data panels. On the right side, a shadowy figure of a cyber rumor-spreader --- a hooded individual in a dark room, face obscured by the glow of multiple monitors --- types frantically, spreading false information across glowing social media interfaces. Between them, digital particles and fragmented text bubbles (\"fake news,\" \"rumor\") drift across the screen like debris. The officers coordinate in sync --- the female officer traces a digital trail with her finger across a glass touchscreen while the male officer initiates a counter-trace protocol. Audio: muffled typing sounds, rapid keystrokes, and a rising digital pulse. Camera performs a slow dolly zoom, creating a subtle disorienting effect that emphasizes the cross-temporal confrontation.\n\n[6s-9s] The female officer executes a decisive gesture --- swiping upward across the main display --- and the screen transforms into a brilliant visual counterattack. A wave of clean blue light erupts from the command center's display, traveling through fiber-optic-like digital pathways across the screen, intercepting and shattering the rumor-spreader's dark red data streams. The male officer leans forward, fingers flying across the keyboard, reinforcing the counter-offensive. The split-screen collapses as the rumor-spreader's monitors flicker and go dark --- the hooded figure recoils, defeated. The officers exchange a quick, triumphant nod. Audio: explosive electronic orchestral swells, triumphant synth stabs, and a resonant bass drop. Rapid-fire editing with sharp cuts between the officers' hands, their focused faces, and the collapsing digital attack.\n\n[9s-12s] The scene transitions to a clean, minimalist studio backdrop. The two officers now stand side by side, facing the camera directly, with professional, authoritative posture. Behind them, the wall seamlessly transforms into a high-definition cinematic typography display --- large bold Chinese characters \"网络不是法外之地\" appear in luminous gold against a deep navy blue gradient background, with subtle particle effects drifting around the text. Audio: The two officers begin a powerful voice-over in clear, authoritative Mandarin Chinese, speaking in unison: \"网络不是法外之地.\" A warm, resonant orchestral chord sustains underneath. Camera slowly pulls back to reveal the full typography layout. Soft golden rim lighting outlines their silhouettes against the dark background.\n\n[12s-15s] The typography transitions --- the first line fades gently as three new lines of text cascade into view: \"不造谣\" , \"不信谣\" , \"不传谣\" --- each appearing in sequence with an elegant cinematic wipe, in crisp white against the deep blue background, accented with subtle golden underlines. Audio: The voice-over continues confidently and concludes: \"不造谣、不信谣、不传谣.\" The two officers deliver a unified, serious yet sincere gaze directly into the lens. The female officer gives a slight, firm nod; the male officer stands with hands clasped behind his back, radiating authority and trust. The camera holds on this final composition --- the officers framed perfectly with the complete typography wall behind them. Audio: The music resolves into a steady, confident closing chord. The frame holds for a beat before fading to black.\n\nNEGATIVE PROMPTS:\nlow quality, worst quality, blurry, pixelated, grainy, noisy, overexposed, underexposed, oversaturated, flat lighting, deformed, distorted, disfigured, malformed, bad anatomy, extra limbs, extra fingers, extra arms, missing limbs, fused fingers, too many fingers, mutated hands, deformed hands, weird hands, asymmetrical face, asymmetrical eyes, unnatural skin texture, plastic skin, static, frozen, jittery, shaky, wobbling, stuttering, jerky, choppy, motion smear, unnatural gait, floating objects, warping, bending, cropped, out of frame, cluttered background, distracting elements, too many people, cartoon, anime, 3d render, CGI, digital illustration, painting, sketch, over-cinematic, text, watermark, logo, subtitle, captions, UI elements, audio noise, audio distortion, glitch, muffled sound",
1344,
768,
15,
435731889644412,
"minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"minimax_h3_video_vae_fp16.safetensors",
"minimax_h3_audio_vae_fp32.safetensors"
],
"widgets_values_named": {
"prompt": "[0s-3s] Medium wide shot of a modern Chinese cybersecurity command center at night. Two young Chinese cyber police officers --- a male and a female, both in sharp dark blue police uniforms with badges and epaulets --- stand before a massive curved LED wall displaying scrolling code, network topology maps, and real-time data streams. The room is dimly lit with cool blue ambient light from the screens, casting soft reflections on their faces. The male officer points at a suspicious data spike on the screen, exchanging a serious glance with his female partner. The screens flicker with red warning indicators. Audio: subtle electronic beeps, keyboard clicks, and low-frequency digital hum. Camera slowly pushes in toward the officers from a slight low angle, building tension.\n\n[3s-6s] A dramatic whip pan transitions to a split-screen composition. On the left side, the two police officers are now seated at a high-tech console, hands moving rapidly across holographic keyboards, eyes locked on floating transparent data panels. On the right side, a shadowy figure of a cyber rumor-spreader --- a hooded individual in a dark room, face obscured by the glow of multiple monitors --- types frantically, spreading false information across glowing social media interfaces. Between them, digital particles and fragmented text bubbles (\"fake news,\" \"rumor\") drift across the screen like debris. The officers coordinate in sync --- the female officer traces a digital trail with her finger across a glass touchscreen while the male officer initiates a counter-trace protocol. Audio: muffled typing sounds, rapid keystrokes, and a rising digital pulse. Camera performs a slow dolly zoom, creating a subtle disorienting effect that emphasizes the cross-temporal confrontation.\n\n[6s-9s] The female officer executes a decisive gesture --- swiping upward across the main display --- and the screen transforms into a brilliant visual counterattack. A wave of clean blue light erupts from the command center's display, traveling through fiber-optic-like digital pathways across the screen, intercepting and shattering the rumor-spreader's dark red data streams. The male officer leans forward, fingers flying across the keyboard, reinforcing the counter-offensive. The split-screen collapses as the rumor-spreader's monitors flicker and go dark --- the hooded figure recoils, defeated. The officers exchange a quick, triumphant nod. Audio: explosive electronic orchestral swells, triumphant synth stabs, and a resonant bass drop. Rapid-fire editing with sharp cuts between the officers' hands, their focused faces, and the collapsing digital attack.\n\n[9s-12s] The scene transitions to a clean, minimalist studio backdrop. The two officers now stand side by side, facing the camera directly, with professional, authoritative posture. Behind them, the wall seamlessly transforms into a high-definition cinematic typography display --- large bold Chinese characters \"网络不是法外之地\" appear in luminous gold against a deep navy blue gradient background, with subtle particle effects drifting around the text. Audio: The two officers begin a powerful voice-over in clear, authoritative Mandarin Chinese, speaking in unison: \"网络不是法外之地.\" A warm, resonant orchestral chord sustains underneath. Camera slowly pulls back to reveal the full typography layout. Soft golden rim lighting outlines their silhouettes against the dark background.\n\n[12s-15s] The typography transitions --- the first line fades gently as three new lines of text cascade into view: \"不造谣\" , \"不信谣\" , \"不传谣\" --- each appearing in sequence with an elegant cinematic wipe, in crisp white against the deep blue background, accented with subtle golden underlines. Audio: The voice-over continues confidently and concludes: \"不造谣、不信谣、不传谣.\" The two officers deliver a unified, serious yet sincere gaze directly into the lens. The female officer gives a slight, firm nod; the male officer stands with hands clasped behind his back, radiating authority and trust. The camera holds on this final composition --- the officers framed perfectly with the complete typography wall behind them. Audio: The music resolves into a steady, confident closing chord. The frame holds for a beat before fading to black.\n\nNEGATIVE PROMPTS:\nlow quality, worst quality, blurry, pixelated, grainy, noisy, overexposed, underexposed, oversaturated, flat lighting, deformed, distorted, disfigured, malformed, bad anatomy, extra limbs, extra fingers, extra arms, missing limbs, fused fingers, too many fingers, mutated hands, deformed hands, weird hands, asymmetrical face, asymmetrical eyes, unnatural skin texture, plastic skin, static, frozen, jittery, shaky, wobbling, stuttering, jerky, choppy, motion smear, unnatural gait, floating objects, warping, bending, cropped, out of frame, cluttered background, distracting elements, too many people, cartoon, anime, 3d render, CGI, digital illustration, painting, sketch, over-cinematic, text, watermark, logo, subtitle, captions, UI elements, audio noise, audio distortion, glitch, muffled sound",
"width": 1344,
"height": 768,
"value_1": 15,
"noise_seed": 435731889644412,
"unet_name": "minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"clip_name": "qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"vae_name": "minimax_h3_video_vae_fp16.safetensors",
"vae_name_1": "minimax_h3_audio_vae_fp32.safetensors"
}
},
{
"id": 116,
"type": "MarkdownNote",
"pos": [
-2080,
4840
],
"size": [
450,
740
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [],
"title": "Note: MiniMax H3",
"properties": {},
"widgets_values": [
"## MiniMax H3\n\n[MiniMax H3](https://www.minimax.io/blog/minimax-h3) is MiniMax's general-purpose, omni-modal generation model. It jointly understands text, image, video, and audio, and generates video with **native stereo audio**: voice, sound effects, and music are modeled jointly in a single forward pass, not layered on afterward. Output is up to 2K resolution, 24fps, and up to about 15 seconds.\n\n## ComfyUI links\n- [ComfyUI#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224)\n- [🤗 Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3)\n\n## About this workflow\n\n**Key inputs**\n\n- **prompt**: describe the shots, camera moves, and the accompanying audio (dialogue, SFX, music) in one block\n- **width / height**: set via Resolution Selector. H3's native canvas is a 768px short edge, capped at 768x1344 pixels, rounded to a multiple of 32\n- **duration (seconds)**: converted to a valid frame `length` by the Math Expression node, snapping up to the model's 17-frame-per-block (17k+5) grid at 24fps\n"
],
"widgets_values_named": {
"text": "## MiniMax H3\n\n[MiniMax H3](https://www.minimax.io/blog/minimax-h3) is MiniMax's general-purpose, omni-modal generation model. It jointly understands text, image, video, and audio, and generates video with **native stereo audio**: voice, sound effects, and music are modeled jointly in a single forward pass, not layered on afterward. Output is up to 2K resolution, 24fps, and up to about 15 seconds.\n\n## ComfyUI links\n- [ComfyUI#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224)\n- [🤗 Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3)\n\n## About this workflow\n\n**Key inputs**\n\n- **prompt**: describe the shots, camera moves, and the accompanying audio (dialogue, SFX, music) in one block\n- **width / height**: set via Resolution Selector. H3's native canvas is a 768px short edge, capped at 768x1344 pixels, rounded to a multiple of 32\n- **duration (seconds)**: converted to a valid frame `length` by the Math Expression node, snapping up to the model's 17-frame-per-block (17k+5) grid at 24fps\n"
},
"color": "#222",
"bgcolor": "#000"
},
{
"id": 117,
"type": "MarkdownNote",
"pos": [
-2550,
4840
],
"size": [
440,
740
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [],
"title": "Note: Model Links",
"properties": {},
"widgets_values": [
"## Model Links\n\n**vae**\n\n- [minimax_h3_video_vae_fp16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors)\n- [minimax_h3_audio_vae_fp32.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors)\n\n**diffusion_models**\n\n- [minimax_h3_fl2va_pruned_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors)\n\n**text_encoders**\n\n- [qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors)\n\n\n## Model Storage Location\n\n```\n📂 ComfyUI/\n├── 📂 models/\n│ ├── 📂 vae/\n│ │ ├── minimax_h3_video_vae_fp16.safetensors\n│ │ └── minimax_h3_audio_vae_fp32.safetensors\n│ ├── 📂 diffusion_models/\n│ │ └── minimax_h3_fl2va_pruned_int8_convrot.safetensors\n│ └── 📂 text_encoders/\n│ └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors\n```\n\n## Report Issue\n\nNote: Please update ComfyUI first ([guide](https://docs.comfy.org/installation/update_comfyui)) and prepare required models. Desktop/Cloud updates follow stable releases, so some nightly-supported models may not be available yet.\n\n- Cannot run / runtime errors: [ComfyUI/issues](https://github.com/comfyanonymous/ComfyUI/issues)\n- UI / frontend issues: [ComfyUI_frontend/issues](https://github.com/Comfy-Org/ComfyUI_frontend/issues)\n- Workflow issues: [workflow_templates/issues](https://github.com/Comfy-Org/workflow_templates/issues)\n"
],
"widgets_values_named": {
"text": "## Model Links\n\n**vae**\n\n- [minimax_h3_video_vae_fp16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors)\n- [minimax_h3_audio_vae_fp32.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors)\n\n**diffusion_models**\n\n- [minimax_h3_fl2va_pruned_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors)\n\n**text_encoders**\n\n- [qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors)\n\n\n## Model Storage Location\n\n```\n📂 ComfyUI/\n├── 📂 models/\n│ ├── 📂 vae/\n│ │ ├── minimax_h3_video_vae_fp16.safetensors\n│ │ └── minimax_h3_audio_vae_fp32.safetensors\n│ ├── 📂 diffusion_models/\n│ │ └── minimax_h3_fl2va_pruned_int8_convrot.safetensors\n│ └── 📂 text_encoders/\n│ └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors\n```\n\n## Report Issue\n\nNote: Please update ComfyUI first ([guide](https://docs.comfy.org/installation/update_comfyui)) and prepare required models. Desktop/Cloud updates follow stable releases, so some nightly-supported models may not be available yet.\n\n- Cannot run / runtime errors: [ComfyUI/issues](https://github.com/comfyanonymous/ComfyUI/issues)\n- UI / frontend issues: [ComfyUI_frontend/issues](https://github.com/Comfy-Org/ComfyUI_frontend/issues)\n- Workflow issues: [workflow_templates/issues](https://github.com/Comfy-Org/workflow_templates/issues)\n"
},
"color": "#222",
"bgcolor": "#000"
},
{
"id": 118,
"type": "MarkdownNote",
"pos": [
-1570,
5460
],
"size": [
300,
520
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [],
"outputs": [],
"title": "Note: Size Settings Reference",
"properties": {},
"widgets_values": [
"| megapixels | Aspect | Output (multiple=32) |\n|---|---|---|\n| 0.2 | 16:9 | 608 x 352 |\n| 0.3 | 16:9 | 736 x 416 |\n| 0.4 | 16:9 | 864 x 480 |\n| 0.5 | 16:9 | 960 x 544 |\n| 0.6 | 16:9 | 1056 x 608 |\n| 0.7 | 16:9 | 1152 x 640 |\n| 0.8 | 16:9 | 1216 x 672 |\n| 0.9 | 16:9 | 1280 x 736 |\n| 0.98 | 16:9 | 1344 x 768 |\n| 1.0 | 16:9 | 1376 x 768 |\n| 1.2 | 16:9 | 1504 x 832 |\n| 1.5 | 16:9 | 1664 x 928 |\n| 1.8 | 16:9 | 1824 x 1024 |\n| 2.0 | 16:9 | 1920 x 1088 |\n"
],
"widgets_values_named": {
"text": "| megapixels | Aspect | Output (multiple=32) |\n|---|---|---|\n| 0.2 | 16:9 | 608 x 352 |\n| 0.3 | 16:9 | 736 x 416 |\n| 0.4 | 16:9 | 864 x 480 |\n| 0.5 | 16:9 | 960 x 544 |\n| 0.6 | 16:9 | 1056 x 608 |\n| 0.7 | 16:9 | 1152 x 640 |\n| 0.8 | 16:9 | 1216 x 672 |\n| 0.9 | 16:9 | 1280 x 736 |\n| 0.98 | 16:9 | 1344 x 768 |\n| 1.0 | 16:9 | 1376 x 768 |\n| 1.2 | 16:9 | 1504 x 832 |\n| 1.5 | 16:9 | 1664 x 928 |\n| 1.8 | 16:9 | 1824 x 1024 |\n| 2.0 | 16:9 | 1920 x 1088 |\n"
},
"color": "#222",
"bgcolor": "#000"
}
],
"links": [
[
194,
105,
0,
92,
0,
"VIDEO"
],
[
219,
115,
0,
105,
2,
"INT"
],
[
220,
115,
1,
105,
3,
"INT"
]
],
"groups": [],
"definitions": {
"subgraphs": [
{
"id": "4c314f31-ecda-4b08-ae98-faaba1bf613f",
"version": 1,
"state": {
"lastGroupId": 4,
"lastNodeId": 122,
"lastLinkId": 238,
"lastRerouteId": 0
},
"revision": 0,
"config": {},
"name": "Image to Video (MiniMax H3)",
"inputNode": {
"id": -10,
"bounding": [
-2560,
4720,
128,
268
]
},
"outputNode": {
"id": -20,
"bounding": [
670,
4780,
128,
68
]
},
"inputs": [
{
"id": "d6eaa195-2266-4016-b7ed-b2d17a3e53c2",
"name": "first_frame",
"type": "IMAGE",
"linkIds": [
195
],
"pos": [
-2456,
4744
]
},
{
"id": "03b95567-f496-4279-9d38-989dd34fa882",
"name": "last_frame",
"type": "IMAGE",
"linkIds": [
196
],
"pos": [
-2456,
4764
]
},
{
"id": "d7302ca7-24ed-44c4-8ac9-736540dab7fb",
"name": "prompt",
"type": "STRING",
"linkIds": [
197
],
"pos": [
-2456,
4784
]
},
{
"id": "709e2d94-3172-496f-ad86-7d672f3df568",
"name": "width",
"type": "INT",
"linkIds": [
200
],
"pos": [
-2456,
4804
]
},
{
"id": "8205b85b-19bb-47f6-8418-89cd42940c1d",
"name": "height",
"type": "INT",
"linkIds": [
201
],
"pos": [
-2456,
4824
]
},
{
"id": "a40e5e96-4307-4a8e-a4f8-13a27d4bc9d3",
"name": "value_1",
"type": "FLOAT",
"linkIds": [
206
],
"label": "duration",
"pos": [
-2456,
4844
]
},
{
"id": "9f8734bc-c2b3-43b0-911e-e89560e8b777",
"name": "noise_seed",
"type": "INT",
"linkIds": [
207
],
"pos": [
-2456,
4864
]
},
{
"id": "9a6d2811-673c-47e0-aa94-dda7c83621e4",
"name": "unet_name",
"type": "COMBO",
"linkIds": [
221
],
"pos": [
-2456,
4884
]
},
{
"id": "7100a787-5536-4002-840b-1d2fd071010a",
"name": "clip_name",
"type": "COMBO",
"linkIds": [
227
],
"pos": [
-2456,
4904
]
},
{
"id": "d40632fd-e0ae-4319-b85d-5479db4d6e11",
"name": "vae_name",
"type": "COMBO",
"linkIds": [
223
],
"pos": [
-2456,
4924
]
},
{
"id": "2992852c-4b20-439a-8771-c866ec1996e6",
"name": "vae_name_1",
"type": "COMBO",
"linkIds": [
224
],
"label": "audio_vae",
"pos": [
-2456,
4944
]
}
],
"outputs": [
{
"id": "adb2611c-e490-4fca-8067-b718c992f8cc",
"name": "VIDEO",
"type": "VIDEO",
"linkIds": [
168
],
"localized_name": "VIDEO",
"pos": [
694,
4804
]
}
],
"widgets": [],
"nodes": [
{
"id": 11,
"type": "VAELoader",
"pos": [
-2020,
4970
],
"size": [
640,
70
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"localized_name": "vae名称",
"name": "vae_name",
"type": "COMBO",
"widget": {
"name": "vae_name"
},
"link": 223
}
],
"outputs": [
{
"localized_name": "VAE",
"name": "VAE",
"type": "VAE",
"links": [
8,
190
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "VAELoader",
"models": [
{
"name": "minimax_h3_video_vae_fp16.safetensors",
"url": "https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors",
"directory": "vae"
}
]
},
"widgets_values": [
"minimax_h3_video_vae_fp16.safetensors"
],
"widgets_values_named": {
"vae_name": "minimax_h3_video_vae_fp16.safetensors"
}
},
{
"id": 24,
"type": "VAELoader",
"pos": [
-2020,
5100
],
"size": [
650,
70
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"localized_name": "vae名称",
"name": "vae_name",
"type": "COMBO",
"widget": {
"name": "vae_name"
},
"link": 224
}
],
"outputs": [
{
"localized_name": "VAE",
"name": "VAE",
"type": "VAE",
"links": [
23
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "VAELoader",
"models": [
{
"name": "minimax_h3_audio_vae_fp32.safetensors",
"url": "https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors",
"directory": "vae"
}
]
},
"widgets_values": [
"minimax_h3_audio_vae_fp32.safetensors"
],
"widgets_values_named": {
"vae_name": "minimax_h3_audio_vae_fp32.safetensors"
}
},
{
"id": 23,
"type": "VAEDecodeAudio",
"pos": [
-50,
4880
],
"size": [
230,
60
],
"flags": {
"collapsed": false
},
"order": 9,
"mode": 0,
"inputs": [
{
"localized_name": "Latent",
"name": "samples",
"type": "LATENT",
"link": 226
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 23
}
],
"outputs": [
{
"localized_name": "音频",
"name": "AUDIO",
"type": "AUDIO",
"links": [
166
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "VAEDecodeAudio"
}
},
{
"id": 10,
"type": "VAEDecode",
"pos": [
-50,
4760
],
"size": [
230,
60
],
"flags": {
"collapsed": false
},
"order": 3,
"mode": 0,
"inputs": [
{
"localized_name": "Latent",
"name": "samples",
"type": "LATENT",
"link": 225
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 8
}
],
"outputs": [
{
"localized_name": "图像",
"name": "IMAGE",
"type": "IMAGE",
"links": [
167
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "VAEDecode"
}
},
{
"id": 17,
"type": "KSamplerSelect",
"pos": [
-372.9545454545457,
5113.409090909091
],
"size": [
370,
70
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"localized_name": "采样器",
"name": "SAMPLER",
"type": "SAMPLER",
"links": [
16
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "KSamplerSelect"
},
"widgets_values": [
"res_multistep"
],
"widgets_values_named": {
"sampler_name": "res_multistep"
}
},
{
"id": 9,
"type": "BasicScheduler",
"pos": [
-758.5958473962723,
5122.484948386696
],
"size": [
370,
130
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"localized_name": "模型",
"name": "model",
"type": "MODEL",
"link": 237
}
],
"outputs": [
{
"localized_name": "Sigmas",
"name": "SIGMAS",
"type": "SIGMAS",
"links": [
18
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "BasicScheduler"
},
"widgets_values": [
"simple",
8,
1
],
"widgets_values_named": {
"scheduler": "simple",
"steps": 8,
"denoise": 1
}
},
{
"id": 14,
"type": "SamplerCustomAdvanced",
"pos": [
-360,
4820
],
"size": [
230,
140
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"localized_name": "噪波",
"name": "noise",
"type": "NOISE",
"link": 40
},
{
"localized_name": "引导器",
"name": "guider",
"type": "GUIDER",
"link": 12
},
{
"localized_name": "采样器",
"name": "sampler",
"type": "SAMPLER",
"link": 16
},
{
"localized_name": "西格玛",
"name": "sigmas",
"type": "SIGMAS",
"link": 18
},
{
"localized_name": "Latent图像",
"name": "latent_image",
"type": "LATENT",
"link": 188
}
],
"outputs": [
{
"localized_name": "Latent",
"name": "output",
"type": "LATENT",
"links": [
225,
226
]
},
{
"localized_name": "降噪Latent",
"name": "denoised_output",
"type": "LATENT",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "SamplerCustomAdvanced"
}
},
{
"id": 16,
"type": "BasicGuider",
"pos": [
-740,
4831.818181818182
],
"size": [
360,
60
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"localized_name": "模型",
"name": "model",
"type": "MODEL",
"link": 236
},
{
"localized_name": "条件",
"name": "conditioning",
"type": "CONDITIONING",
"link": 187
}
],
"outputs": [
{
"localized_name": "引导器",
"name": "GUIDER",
"type": "GUIDER",
"links": [
12
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "BasicGuider"
}
},
{
"id": 6,
"type": "UNETLoader",
"pos": [
-2020,
4630
],
"size": [
640,
90
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"localized_name": "UNet名称",
"name": "unet_name",
"type": "COMBO",
"widget": {
"name": "unet_name"
},
"link": 221
}
],
"outputs": [
{
"localized_name": "模型",
"name": "MODEL",
"type": "MODEL",
"links": [
233
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "UNETLoader",
"models": [
{
"name": "minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"url": "https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"directory": "diffusion_models"
}
]
},
"widgets_values": [
"minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"default"
],
"widgets_values_named": {
"unet_name": "minimax_h3_fl2va_pruned_int8_convrot.safetensors",
"weight_dtype": "default"
}
},
{
"id": 13,
"type": "CLIPLoader",
"pos": [
-2020,
4780
],
"size": [
640,
120
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"localized_name": "CLIP名称",
"name": "clip_name",
"type": "COMBO",
"widget": {
"name": "clip_name"
},
"link": 227
}
],
"outputs": [
{
"localized_name": "CLIP",
"name": "CLIP",
"type": "CLIP",
"links": [
189
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "CLIPLoader",
"models": [
{
"name": "qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"url": "https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"directory": "text_encoders"
}
]
},
"widgets_values": [
"qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"minimax",
"default"
],
"widgets_values_named": {
"clip_name": "qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
"type": "minimax",
"device": "default"
}
},
{
"id": 15,
"type": "RandomNoise",
"pos": [
-790,
4660
],
"size": [
360,
90
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"localized_name": "噪波随机种",
"name": "noise_seed",
"type": "INT",
"widget": {
"name": "noise_seed"
},
"link": 207
}
],
"outputs": [
{
"localized_name": "噪波",
"name": "NOISE",
"type": "NOISE",
"links": [
40
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "RandomNoise"
},
"widgets_values": [
1,
"randomize"
],
"widgets_values_named": {
"noise_seed": 1,
"control_after_generate": "randomize"
}
},
{
"id": 91,
"type": "CreateVideo",
"pos": [
260,
4790
],
"size": [
270,
110
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"localized_name": "图像",
"name": "images",
"type": "IMAGE",
"link": 167
},
{
"localized_name": "音频",
"name": "audio",
"shape": 7,
"type": "AUDIO",
"link": 166
}
],
"outputs": [
{
"localized_name": "视频",
"name": "VIDEO",
"type": "VIDEO",
"links": [
168
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "CreateVideo"
},
"widgets_values": [
24,
8
],
"widgets_values_named": {
"fps": 24,
"bit_depth": 8
}
},
{
"id": 104,
"type": "MiniMaxH3ImageToVideo",
"pos": [
-1277.4999999999998,
5057.159090909093
],
"size": [
410,
510
],
"flags": {},
"order": 12,
"mode": 0,
"inputs": [
{
"localized_name": "clip",
"name": "clip",
"type": "CLIP",
"link": 189
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 190
},
{
"localized_name": "first_frame",
"name": "first_frame",
"shape": 7,
"type": "IMAGE",
"link": 195
},
{
"localized_name": "last_frame",
"name": "last_frame",
"shape": 7,
"type": "IMAGE",
"link": 196
},
{
"localized_name": "prompt",
"name": "prompt",
"type": "STRING",
"widget": {
"name": "prompt"
},
"link": 197
},
{
"localized_name": "width",
"name": "width",
"type": "INT",
"widget": {
"name": "width"
},
"link": 200
},
{
"localized_name": "height",
"name": "height",
"type": "INT",
"widget": {
"name": "height"
},
"link": 201
},
{
"localized_name": "length",
"name": "length",
"type": "INT",
"widget": {
"name": "length"
},
"link": 199
}
],
"outputs": [
{
"localized_name": "positive",
"name": "positive",
"type": "CONDITIONING",
"links": [
187
]
},
{
"localized_name": "Latent",
"name": "LATENT",
"type": "LATENT",
"links": [
188
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "MiniMaxH3ImageToVideo"
},
"widgets_values": [
"Vaporwave title sequence look: pink and blue gradient palette, VHS tracking artifacts, Greek statue motifs, chrome palm trees, RGB chromatic aberration, lo-fi retro atmosphere, mood languid and nostalgic.\n\nTimeline:\n[0s-1s] VHS static opens the frame, the title \"COMFYUI\" appears with RGB split and a slight horizontal jitter.\n[1s-2.5s] Hard cut, a Greek plaster bust close-up, pink-purple gradient sky, a pixelated sun.\n[2.5s-4s] Clean \"STARRING\" credits appear, \"LATENT\" and \"CONTROLNET\" each shown exactly once.\n[4s-5s] Final card \"DIRECTED BY COMFYUI\" holds, one VHS tracking glitch settling into stability.\n\nHard cuts only, transitions landing with tape jumps, no push-ins, no dissolves.\n\nAudio: lo-fi vaporwave score, slow drum machine with soft bass, VHS tape-noise sample joins at 2.5s, melody fading for the last 1s.\n\nAll text must be clearly legible, do not misspell English, no Chinese characters, do not repeat names or job titles, no soft dissolves, no subtitle bars.",
1344,
768,
73
],
"widgets_values_named": {
"prompt": "Vaporwave title sequence look: pink and blue gradient palette, VHS tracking artifacts, Greek statue motifs, chrome palm trees, RGB chromatic aberration, lo-fi retro atmosphere, mood languid and nostalgic.\n\nTimeline:\n[0s-1s] VHS static opens the frame, the title \"COMFYUI\" appears with RGB split and a slight horizontal jitter.\n[1s-2.5s] Hard cut, a Greek plaster bust close-up, pink-purple gradient sky, a pixelated sun.\n[2.5s-4s] Clean \"STARRING\" credits appear, \"LATENT\" and \"CONTROLNET\" each shown exactly once.\n[4s-5s] Final card \"DIRECTED BY COMFYUI\" holds, one VHS tracking glitch settling into stability.\n\nHard cuts only, transitions landing with tape jumps, no push-ins, no dissolves.\n\nAudio: lo-fi vaporwave score, slow drum machine with soft bass, VHS tape-noise sample joins at 2.5s, melody fading for the last 1s.\n\nAll text must be clearly legible, do not misspell English, no Chinese characters, do not repeat names or job titles, no soft dissolves, no subtitle bars.",
"width": 1344,
"height": 768,
"length": 73
}
},
{
"id": 107,
"type": "ComfyMathExpression",
"pos": [
-1724.7727272727273,
5519.318181818182
],
"size": [
360,
160
],
"flags": {
"collapsed": false
},
"order": 13,
"mode": 0,
"inputs": [
{
"label": "a",
"localized_name": "values.a",
"name": "values.a",
"type": "FLOAT,INT,BOOLEAN",
"link": 205
},
{
"label": "b",
"localized_name": "values.b",
"name": "values.b",
"shape": 7,
"type": "FLOAT,INT,BOOLEAN",
"link": null
}
],
"outputs": [
{
"localized_name": "浮点",
"name": "FLOAT",
"type": "FLOAT",
"links": null
},
{
"localized_name": "整数",
"name": "INT",
"type": "INT",
"links": [
199
]
},
{
"localized_name": "布尔值",
"name": "BOOL",
"type": "BOOLEAN",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "ComfyMathExpression"
},
"widgets_values": [
"max(5, round(a * 24)) + (5 - (max(5, round(a * 24)) % 17)) % 17"
],
"widgets_values_named": {
"expression": "max(5, round(a * 24)) + (5 - (max(5, round(a * 24)) % 17)) % 17"
}
},
{
"id": 111,
"type": "PrimitiveFloat",
"pos": [
-2020,
5300
],
"size": [
270,
70
],
"flags": {},
"order": 14,
"mode": 0,
"inputs": [
{
"localized_name": "值",
"name": "value",
"type": "FLOAT",
"widget": {
"name": "value"
},
"link": 206
}
],
"outputs": [
{
"localized_name": "浮点",
"name": "FLOAT",
"type": "FLOAT",
"links": [
205
]
}
],
"title": "Float (duration)",
"properties": {
"cnr_id": "comfy-core",
"ver": "0.30.0",
"Node name for S&R": "PrimitiveFloat"
},
"widgets_values": [
2
],
"widgets_values_named": {
"value": 2
}
},
{
"id": 121,
"type": "TESpeedMiniMaxH3",
"pos": [
-1233.3421246652763,
4629.632227273776
],
"size": [
306.640625,
226
],
"flags": {},
"order": 15,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 233
}
],
"outputs": [
{
"localized_name": "模型",
"name": "MODEL",
"type": "MODEL",
"links": [
238
]
}
],
"properties": {
"aux_id": "HELPMEEADICE/TE-Speed-MiniMaxH3-OSS",
"ver": "c1dacf47bc02cb9326f7b93c69280529b93d391b",
"Node name for S&R": "TESpeedMiniMaxH3"
},
"widgets_values": [
0.12,
0.1,
0.9,
2,
"gpu",
0.75
],
"widgets_values_named": {
"processing_control_value": 0.12,
"processing_percent_1": 0.1,
"processing_percent_2": 0.9,
"mcs": 2,
"device": "gpu",
"cache_depth": 0.75
}
},
{
"id": 122,
"type": "LoraLoaderModelOnly",
"pos": [
-1063.9082068918183,
4903.27793506344
],
"size": [
270,
90
],
"flags": {},
"order": 16,
"mode": 0,
"inputs": [
{
"localized_name": "模型",
"name": "model",
"type": "MODEL",
"link": 238
}
],
"outputs": [
{
"localized_name": "模型",
"name": "MODEL",
"type": "MODEL",
"links": [
236,
237
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.33.0",
"Node name for S&R": "LoraLoaderModelOnly"
},
"widgets_values": [
"minimax_h3_turbo_v4_step600_ema.safetensors",
1
],
"widgets_values_named": {
"lora_name": "minimax_h3_turbo_v4_step600_ema.safetensors",
"strength_model": 1
}
}
],
"groups": [
{
"id": 1,
"title": "Models",
"bounding": [
-2050,
4540,
700,
670
],
"color": "#3f789e",
"flags": {}
},
{
"id": 2,
"title": "Sampling",
"bounding": [
-810,
4540,
690,
670
],
"color": "#3f789e",
"flags": {}
},
{
"id": 3,
"title": "Conditioning",
"bounding": [
-1320,
4540,
480,
670
],
"color": "#3f789e",
"flags": {}
},
{
"id": 4,
"title": "Decoding and create video",
"bounding": [
-90,
4540,
670,
670
],
"color": "#3f789e",
"flags": {}
}
],
"links": [
{
"id": 23,
"origin_id": 24,
"origin_slot": 0,
"target_id": 23,
"target_slot": 1,
"type": "VAE"
},
{
"id": 8,
"origin_id": 11,
"origin_slot": 0,
"target_id": 10,
"target_slot": 1,
"type": "VAE"
},
{
"id": 40,
"origin_id": 15,
"origin_slot": 0,
"target_id": 14,
"target_slot": 0,
"type": "NOISE"
},
{
"id": 12,
"origin_id": 16,
"origin_slot": 0,
"target_id": 14,
"target_slot": 1,
"type": "GUIDER"
},
{
"id": 16,
"origin_id": 17,
"origin_slot": 0,
"target_id": 14,
"target_slot": 2,
"type": "SAMPLER"
},
{
"id": 18,
"origin_id": 9,
"origin_slot": 0,
"target_id": 14,
"target_slot": 3,
"type": "SIGMAS"
},
{
"id": 188,
"origin_id": 104,
"origin_slot": 1,
"target_id": 14,
"target_slot": 4,
"type": "LATENT"
},
{
"id": 187,
"origin_id": 104,
"origin_slot": 0,
"target_id": 16,
"target_slot": 1,
"type": "CONDITIONING"
},
{
"id": 167,
"origin_id": 10,
"origin_slot": 0,
"target_id": 91,
"target_slot": 0,
"type": "IMAGE"
},
{
"id": 166,
"origin_id": 23,
"origin_slot": 0,
"target_id": 91,
"target_slot": 1,
"type": "AUDIO"
},
{
"id": 189,
"origin_id": 13,
"origin_slot": 0,
"target_id": 104,
"target_slot": 0,
"type": "CLIP"
},
{
"id": 190,
"origin_id": 11,
"origin_slot": 0,
"target_id": 104,
"target_slot": 1,
"type": "VAE"
},
{
"id": 168,
"origin_id": 91,
"origin_slot": 0,
"target_id": -20,
"target_slot": 0,
"type": "VIDEO"
},
{
"id": 195,
"origin_id": -10,
"origin_slot": 0,
"target_id": 104,
"target_slot": 2,
"type": "IMAGE"
},
{
"id": 196,
"origin_id": -10,
"origin_slot": 1,
"target_id": 104,
"target_slot": 3,
"type": "IMAGE"
},
{
"id": 197,
"origin_id": -10,
"origin_slot": 2,
"target_id": 104,
"target_slot": 4,
"type": "STRING"
},
{
"id": 199,
"origin_id": 107,
"origin_slot": 1,
"target_id": 104,
"target_slot": 7,
"type": "INT"
},
{
"id": 200,
"origin_id": -10,
"origin_slot": 3,
"target_id": 104,
"target_slot": 5,
"type": "INT"
},
{
"id": 201,
"origin_id": -10,
"origin_slot": 4,
"target_id": 104,
"target_slot": 6,
"type": "INT"
},
{
"id": 205,
"origin_id": 111,
"origin_slot": 0,
"target_id": 107,
"target_slot": 0,
"type": "FLOAT"
},
{
"id": 206,
"origin_id": -10,
"origin_slot": 5,
"target_id": 111,
"target_slot": 0,
"type": "FLOAT"
},
{
"id": 207,
"origin_id": -10,
"origin_slot": 6,
"target_id": 15,
"target_slot": 0,
"type": "INT"
},
{
"id": 221,
"origin_id": -10,
"origin_slot": 7,
"target_id": 6,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 223,
"origin_id": -10,
"origin_slot": 9,
"target_id": 11,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 224,
"origin_id": -10,
"origin_slot": 10,
"target_id": 24,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 225,
"origin_id": 14,
"origin_slot": 0,
"target_id": 10,
"target_slot": 0,
"type": "LATENT"
},
{
"id": 226,
"origin_id": 14,
"origin_slot": 0,
"target_id": 23,
"target_slot": 0,
"type": "LATENT"
},
{
"id": 227,
"origin_id": -10,
"origin_slot": 8,
"target_id": 13,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 233,
"origin_id": 6,
"origin_slot": 0,
"target_id": 121,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 236,
"origin_id": 122,
"origin_slot": 0,
"target_id": 16,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 237,
"origin_id": 122,
"origin_slot": 0,
"target_id": 9,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 238,
"origin_id": 121,
"origin_slot": 0,
"target_id": 122,
"target_slot": 0,
"type": "MODEL"
}
],
"extra": {}
}
]
},
"config": {},
"extra": {
"ds": {
"scale": 1.0138731383022266,
"offset": [
1649.2058698857886,
-4571.467245411632
]
},
"frontendVersion": "1.49.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}6.4 运行日志
bash
./run-优化.sh
[INFO] setup plugin alembic.autogenerate.schemas
[INFO] setup plugin alembic.autogenerate.tables
[INFO] setup plugin alembic.autogenerate.types
[INFO] setup plugin alembic.autogenerate.constraints
[INFO] setup plugin alembic.autogenerate.defaults
[INFO] setup plugin alembic.autogenerate.comments
[INFO] setup plugin alembic.autogenerate.checkconstraint_byname
[WARNING] Could not autodetect AIMDO implementation, assuming Nvidia
[START] Security scan
[INFO] [ComfyUI-Manager] Using `uv` as Python module for pip operations.
Using Python 3.10.12 environment at: /usr
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2026-08-23 16:07:35.880
** Platform: Linux
** Python version: 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0]
** Python executable: /usr/bin/python
** ComfyUI Path: /home/models/FireRed-Image-Edit-1.1-ComfyUI/ComfyUI-master
** ComfyUI Base Folder Path: /home/models/FireRed-Image-Edit-1.1-ComfyUI/ComfyUI-master
** User directory: /home/models/ComfyUI-master/user
** ComfyUI-Manager config path: /home/models/ComfyUI-master/user/__manager/config.ini
** Log path: /home/models/ComfyUI-master/user/comfyui.log
Using Python 3.10.12 environment at: /usr
Using Python 3.10.12 environment at: /usr
[INFO]
Prestartup times for custom nodes:
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/rgthree-comfy
[INFO] 0.8 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-Manager
[INFO]
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_embedding', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'int8_linear', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend hip: {'available': False, 'disabled': False, 'unavailable_reason': 'Failed to load HIP extension: libamdhip64.so.7: cannot open shared object file: No such file or directory', 'capabilities': []}
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'int8_linear', 'na3d', 'quantize_and_rotate_rowwise', 'quantize_int8_rowwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'w4a8_int8_linear']}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 65520 MB, total RAM 515687 MB
[INFO] pytorch version: 2.10.0
[INFO] Set: torch.backends.cudnn.enabled = False for better AMD performance.
[INFO] AMD arch: gfx928
[INFO] ROCm version: (6, 3)
/home/models/ComfyUI-master/comfy/model_management.py:506: UserWarning: expandable_segments not supported on this platform (Triggered internally at /pytorch/c10/hip/HIPAllocatorConfig.h:42.)
q = torch.empty((1, 1, 8, 64), dtype=torch.float16, device=get_torch_device())
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 K100_AI : native
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 464118.0
[INFO] Flash Attention backend: native
[INFO] Native Flash Attention QKV repack: enabled
[INFO] Using Flash Attention
/usr/local/lib/python3.10/dist-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
setattr(self, word, getattr(machar, word).flat[0])
/usr/local/lib/python3.10/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
return self._float_to_str(self.smallest_subnormal)
/usr/local/lib/python3.10/dist-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
setattr(self, word, getattr(machar, word).flat[0])
/usr/local/lib/python3.10/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
return self._float_to_str(self.smallest_subnormal)
[INFO] Python version: 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0]
[INFO] ComfyUI version: 0.33.0
[INFO] comfy-aimdo version: 0.4.13
[INFO] comfy-kitchen version: 0.2.31
[INFO] comfyui-frontend-package version: 1.49.6
[INFO] comfyui-workflow-templates version: 0.11.44
[INFO] comfyui-embedded-docs version: 0.5.10
[INFO] comfy-kitchen version: 0.2.31
[INFO] comfy-aimdo version: 0.4.13
[INFO] [Prompt Server] web root: /usr/local/lib/python3.10/dist-packages/comfyui_frontend_package/static
[INFO] Asset seeder disabled
[INFO] No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
[rgthree-comfy] Loaded 48 epic nodes. 🎉
[rgthree-comfy] ComfyUI's new Node 2.0 rendering may be incompatible with some rgthree-comfy nodes and features, breaking some rendering as well as losing the ability to access a node's properties (a vital part of many nodes). It also appears to run MUCH more slowly spiking CPU usage and causing jankiness and unresponsiveness, especially with large workflows. Personally I am not planning to use the new Nodes 2.0 and, unfortunately, am not able to invest the time to investigate and overhaul rgthree-comfy where needed. If you have issues when Nodes 2.0 is enabled, I'd urge you to switch it off as well and join me in hoping ComfyUI is not planning to deprecate the existing, stable canvas rendering all together.
[WARNING] Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0).
WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `/home/models/ComfyUI-master/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 220 nodes successfully.
"Believe in yourself, and the rest will fall into place." - Unknown
[INFO] ### Loading: ComfyUI-Manager (V3.41)
[INFO] [ComfyUI-Manager] network_mode: public
[INFO] [ComfyUI-Manager] ComfyUI per-queue preview override detected (PR #11261). Manager's preview method feature is disabled. Use ComfyUI's --preview-method CLI option or 'Settings > Execution > Live preview method'.
[INFO] ### ComfyUI Version: v0.33.0-40-g783545f68 | Released on '2026-08-22'
Unable to import `torchao` Tensor objects. This may affect loading checkpoints serialized with `torchao`
[INFO] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
aiter failed to import: cannot import name 'flash_attn_func' from 'aiter' (/usr/local/lib/python3.10/dist-packages/aiter/__init__.py). Falling back to native attention.
[FlashVSR] Loading nodes...
[FlashVSR] ✓ SageAttention detected (~20-30% speedup enabled)
[FlashVSR] ✓ Loaded 2 node(s)
[INFO] [MultiGPU Core Patching] Patching mm.soft_empty_cache for Comprehensive Memory Management (VRAM + CPU + Store Pruning)
[MultiGPU Core Patching] Patching mm.get_torch_device, mm.text_encoder_device, mm.unet_offload_device
[MultiGPU DEBUG] Initial current_device: cuda:0
[MultiGPU DEBUG] Initial current_text_encoder_device: cuda:0
[MultiGPU DEBUG] Initial current_unet_offload_device: cpu
[MultiGPU] Patched comfy.model_management.current_stream to honor CUDA device arguments
[MultiGPU] Patched comfy.sample.sample with runtime device guard
[MultiGPU] Patched comfy.sample.sample_custom with runtime device guard
[MultiGPU] Applied comfy_kitchen CUDA DLPack device guard patch (P2P-aware)
[MultiGPU] DynamicVRAM not enabled; skipping multi-device aimdo initialization
[MultiGPU] Initiating custom_node Registration. . .
-----------------------------------------------
custom_node Found Nodes
-----------------------------------------------
ComfyUI-LTXVideo N 0
ComfyUI-Florence2 N 0
ComfyUI_bitsandbytes_NF4 N 0
x-flux-comfyui N 0
ComfyUI-MMAudio N 0
ComfyUI-GGUF N 0
PuLID_ComfyUI N 0
ComfyUI-WanVideoWrapper N 0
-----------------------------------------------
[MultiGPU] Registration complete. Final mappings: CheckpointLoaderAdvancedMultiGPU, CheckpointLoaderAdvancedDisTorch2MultiGPU, DeviceSelectorMultiGPU, UNetLoaderLP, UNETLoaderMultiGPU, VAELoaderMultiGPU, CLIPLoaderMultiGPU, DualCLIPLoaderMultiGPU, TripleCLIPLoaderMultiGPU, QuadrupleCLIPLoaderMultiGPU, CLIPVisionLoaderMultiGPU, CheckpointLoaderSimpleMultiGPU, ControlNetLoaderMultiGPU, DiffusersLoaderMultiGPU, DiffControlNetLoaderMultiGPU, UNETLoaderDisTorch2MultiGPU, VAELoaderDisTorch2MultiGPU, CLIPLoaderDisTorch2MultiGPU, DualCLIPLoaderDisTorch2MultiGPU, TripleCLIPLoaderDisTorch2MultiGPU, QuadrupleCLIPLoaderDisTorch2MultiGPU, CLIPVisionLoaderDisTorch2MultiGPU, CheckpointLoaderSimpleDisTorch2MultiGPU, ControlNetLoaderDisTorch2MultiGPU, DiffusersLoaderDisTorch2MultiGPU, DiffControlNetLoaderDisTorch2MultiGPU
USDU batch patches applied successfully.
[INFO] USDU batch patches applied successfully.
/home/models/ComfyUI-master/custom_nodes/raylight/../ComfyUI-GGUF
/home/models/ComfyUI-master/custom_nodes/ComfyUI-GGUF
City96 GGUF not found, GGUF ray loader disable
[INFO] ComfyUI-GGUF not found, using our implementation
[ROCm Ninodes] Successfully loaded from rocm_nodes package
[INFO]
Import times for custom nodes:
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/websocket_image_save.py
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-MiniMaxH3DualClockSampler
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/TE-Speed-MiniMaxH3-OSS
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-TiledDiffusion
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-minimax-h3-turbo
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-ParallelAnything
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-RuiquNodes
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-post-processing-nodes
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-speed-minimaxH3
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-yaser-nodes
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/pseudocomfy
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/rocm-ninodes
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-MiniMax-H3-LongMedia
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui_ultimatesdupscale
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-FlashVSR
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/rgthree-comfy
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-MultiGPU
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-videohelpersuite
[INFO] 0.0 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-kjnodes
[INFO] 0.2 seconds: /home/models/ComfyUI-master/custom_nodes/ComfyUI-Manager
[INFO] 0.2 seconds: /home/models/ComfyUI-master/custom_nodes/raylight
[INFO] 1.9 seconds: /home/models/ComfyUI-master/custom_nodes/was-node-suite-comfyui
[INFO] 2.6 seconds: /home/models/ComfyUI-master/custom_nodes/comfyui-firered-trent
[INFO]
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Using RAM pressure cache.
[INFO] Starting server
[INFO] To see the GUI go to: http://0.0.0.0:7681
[INFO] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[INFO] got prompt
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] Found quantization metadata version 1
[MultiGPU Core Patching] text_encoder_device_patched returning device: cuda:0 (current_text_encoder_device=cuda:0)
[INFO] Using MixedPrecisionOps for text encoder
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load MiniMaxH3TEModel_
[INFO] loaded completely; 63338.80 MB usable, 14960.20 MB loaded, full load: True
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: asym_w4a8_int8, int8_tensorwise, convrot_w4a4 , emulated ops: float8_e4m3fn, float8_e5m2, nvfp4, mxfp8
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW_AV
[INFO] Requested to load MiniMaxH3
[INFO] 0 models unloaded.
[ERROR] ERROR lora diffusion_model.blocks.9.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.8.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.7.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.6.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.5.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.48.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.47.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.46.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.45.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.44.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.43.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.42.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.41.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.40.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.4.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.39.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.38.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.37.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.36.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.35.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.34.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.33.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.32.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.31.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.30.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.3.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.29.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.28.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.27.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.26.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.25.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.24.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.23.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.22.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.21.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.20.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.2.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.19.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.18.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.17.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.16.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.15.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.14.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.13.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.12.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.11.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.10.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.1.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.0.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.final_layer.adaln_proj.linear.weight shape '[10752, 8]' is invalid for input of size 28901376
[INFO] loaded completely; 25507.20 MB usable, 19996.14 MB loaded, full load: True
FETCH DATA from: /home/models/ComfyUI-master/custom_nodes/ComfyUI-Manager/custom-node-list.json [DONE]
[INFO] [ComfyUI-Manager] All startup tasks have been completed.
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [21:50<00:00, 163.84s/it]
[INFO] Requested to load MiniMaxH3AudioVAE
[INFO] loaded completely; 39861.50 MB usable, 577.08 MB loaded, full load: True
[INFO] Requested to load MiniMaxH3VideoVAE
[INFO] loaded completely; 38960.19 MB usable, 4966.19 MB loaded, full load: True
[INFO] Prompt executed in 00:28:04
[INFO] got prompt
[INFO] 0 models unloaded.
0%| | 0/8 [00:00<?, ?it/s]TE-Speed-MiniMaxH3(OSS): acceleration 38.0% (full=4 cache=4 of 8 steps, skipped 152/400 blocks)
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [21:49<00:00, 163.68s/it]
[INFO] Requested to load MiniMaxH3AudioVAE
[INFO] loaded completely; 39775.50 MB usable, 577.08 MB loaded, full load: True
[INFO] Requested to load MiniMaxH3VideoVAE
[INFO] loaded completely; 38892.19 MB usable, 4966.19 MB loaded, full load: True
[INFO] Prompt executed in 00:27:19minimax_h3_fl2va_pruned_int8_convrot.safetensors 是 MiniMax-H3 视频生成模型的一个专为消费级显卡优化的、经过剪枝（Pruned）和 INT8 量化的官方版本，以上日志中出现ERROR错误是因为这个模型导致的，但是不影响视频生成及视频质量！
七、完整加速链路总结
7.1 端到端加速流水线
整个推理过程的执行路径及每个环节的加速策略如下：
7.2 各组件加速效益对照表
| 优化层级 | 组件/操作 | 加速贡献 | 实施难度 | 失效风险 |
|---|---|---|---|---|
| 模型层 | INT8量化H3主模型 | 显存占用↓50% | 低（直接下载） | 无 |
| 4步Turbo LoRA | 采样步数↓70%+ | 低（加载即用） | 需配合TE-Speed调度 | |
| 算子层 | SageAttention 1.0.6 | ~11% | 低（pip安装） | 版本必须锁定 |
| attention.py底层适配 | ~25%~30%（vs SDPA回退） | 高（需理解硬件） | 核心依赖，不可省略 | |
| 调度层 | TE-Speed块级缓存 | ~45% | 中（需patch_model） | 需新版ComfyUI支持 |
| 系统层 | 8实例并行 | 8×吞吐 | 低（脚本启动） | 需足够显存（64GB/卡 |
7.3 配置决策树：为什么是这个组合？
面对大量可选优化方案，本次调优的配置决策遵循以下逻辑：
bash
问题：33B模型在64GB显存上推理速度慢
│
├─ 首要瓶颈：显存带宽（896GB/s）被注意力计算占满
│ └─ 解决方案：Flash Attention + SageAttention
│ └─ 但K100_AI上Triton默认配置会崩溃
│ └─ attention.py硬锁定BLOCK_M=64 ← 必须修改
│
├─ 次要瓶颈：50层DiT逐层串行计算
│ └─ 解决方案：TE-Speed块级缓存（残差复用）
│ └─ 条件：相邻步sigma变化<0.12时才触发
│ └─ 配合Turbo LoRA（4步） → 缓存命中率极高
│
└─ 系统级瓶颈：单卡生成耗时22分钟/视频
└─ 解决方案：8卡并行
└─ 前提：每卡独立加载模型（64GB × 8 = 512GB显存池）7.4 最终效果
| 指标 | 优化前 | 优化后 | 提升幅度 |
|---|---|---|---|
| 单卡15秒高清视频（768×1376） | 无法稳定运行（OOM/崩溃） | ~22分钟 | 从不可用到可用 |
| 单卡15秒标准视频（0.5M像素） | 约45分钟（回退SDPA） | ~7分钟 | 6.4倍 |
| 8卡并行吞吐 | 不适用 | 20分钟/120秒高清视频 | 8×线性扩展 |
7.5 关键经验
-
国产硬件适配的优先级高于算法优化 ：在K100_AI上，让Flash Attention"能跑"比"跑得快"更重要。
attention.py的修改虽然牺牲了部分理论峰值性能，却换来了100%的执行确定性。 -
版本锁定是隐形的性能护城河：SageAttention 1.0.6、DTK 2604、flash-attn 2.8.x这一组合是经过大量试错后稳定的"配方"，任何单一版本变动都可能导致崩溃或性能回退。
-
块级缓存是DiT模型推理的"杀手锏"：TE-Speed带来的45%提速远超单一算子优化，因为它利用的是扩散模型采样过程中相邻步隐空间变化平滑的数学本质。
八、实测数据
| 视频规格 | 分辨率 | 时长 | 单卡耗时 | 8卡并行耗时 |
|---|---|---|---|---|
| 标准画质 | 0.5M像素 | 15秒 | ~7分钟 | ~7分钟（8个） |
| 高清画质 | 768×1376 (1M像素) | 15秒 | ~22分钟 | ~20分钟（8个） |
8卡并行总产出 ：约20分钟生成120秒768P高清视频。
ComfyUI工作流截图
九、踩坑避坑指南（血泪教训十则）
纸上得来终觉浅，绝知此事要躬行。以下十点是本人在海光K100_AI上调优ComfyUI+MiniMax-H3过程中，踩过的最深、最隐蔽的"坑"以及对应的逃生方案。每一条都曾导致数小时的无效调试，希望后来者能绕道而行。
避坑 1：Docker 启动遗漏设备挂载，导致 DCU 不可见
-
症状 ：容器内执行
rocm-smi或hipconfig无法识别显卡，ComfyUI 启动后提示CUDA device not available或直接回退到 CPU 推理（速度极慢）。 -
根本原因 ：海光 DCU 的设备文件（
/dev/kfd、/dev/dri、/dev/mkfd）未正确映射到容器内部。其中/dev/mkfd是海光特有的多卡管理设备，遗漏后 HIP 运行时无法初始化。 -
正确做法 ：严格按照本文的
docker run命令，一次性挂载全部三个设备文件，并添加--group-add video和--cap-add=SYS_PTRACE。缺一不可。 -
验证命令 ：进入容器后执行
ls -la /dev/kfd /dev/dri /dev/mkfd，确保三个文件均存在且权限正确。
避坑 2：直接 pip install torch 覆盖了 DTK 定制的 PyTorch
-
症状 ：
pip install -r requirements.txt后，ComfyUI 启动时报HIP error: invalid device function或no kernel image is available for execution on the device。 -
根本原因 ：ComfyUI 官方
requirements.txt中包含了torch、torchvision、torchaudio依赖。默认 PyPI 源会下载 CUDA 版本的 PyTorch，覆盖了基础镜像中已预置的 DTK 优化版 PyTorch，导致内核与海光 DCU 指令集不匹配。 -
正确做法 ：安装 ComfyUI 前 ，务必手动删除
requirements.txt中所有torch开头的行（本文已提供删改后的内容）。安装完其他依赖后，通过pip list | grep torch确认版本号中包含+dtk后缀。
避坑 3：SageAttention 版本"追新"导致 illegal memory access
-
症状 ：启动
--use-sage-attention后，采样刚开始几秒即崩溃，控制台输出CUDA illegal memory access或hipError_t错误码，且错误堆栈指向sageattention内核。 -
根本原因：SageAttention 2.2.0 及以上版本重构了 Flash Attention 内核调度逻辑，与 MiniMax-H3 中非标准的 FP8/NF4 精度层存在隐性冲突。在海光 DTK 平台上，这种冲突会直接触发显存越界。
-
正确做法 ：版本锁定是铁律 。执行
pip install sageattention==1.0.6，不升级、不降级。建议在requirements.txt中显式写入版本约束。
避坑 4：Triton 自动调优（Autotune）引发 VM Fault 崩溃
-
症状 ：启用 Flash Attention 后，短文本（如 1~2 秒视频）可正常生成，但生成 10 秒以上视频时，控制台输出
VM Fault、Page Fault或LDS allocation exceeded，内核直接终止。 -
根本原因 ：
flash-attn的 Triton 后端会根据序列长度自动选择最优分块参数。当序列较长时，Triton 倾向于选择BLOCK_M=256、num_warps=8，所需共享内存（LDS）超过 K100_AI 的 64KB 上限，触发硬件级页错误。 -
正确做法 ：必须应用本文
attention.py的修改 ，将 Triton 配置列表硬截断为BLOCK_M=64、BLOCK_N=64、num_warps=4。不要抱有侥幸心理尝试调整阈值，64 是经过长序列（56,448 tokens）验证的安全值。 -
验证方法 ：启动时观察日志
Flash Attention backend: native（若使用 Native 后端则无此问题），若使用 Triton 后端则必须确认锁频生效。
避坑 5：TE-Speed 与旧版 ComfyUI 不兼容，导致采样步数错乱
-
症状 ：安装 TE-Speed 并执行
patch_model.py后，ComfyUI 启动正常，但采样器实际执行的步数与设置不符，或生成视频出现严重的画面跳帧/残影。 -
根本原因 ：TE-Speed 通过钩子（Hook）劫持了 DiT 模型的
block_loop。新版 ComfyUI（v0.30+）重构了 MiniMax-H3 的执行调度机制，旧版 TE-Speed 的钩子插入位置偏移，导致残差缓存逻辑失效或作用于错误的块索引。 -
正确做法：
-
确保 ComfyUI 版本为 最新 master 分支（至少支持本文工作流中的 v0.30.0 架构）。
-
TE-Speed 必须使用 3.2 及以上版本 ，并严格按照
python patch_model.py --comfy-ui <路径>执行补丁。 -
回滚命令 ：若出现问题，执行
python patch_model.py --revert可快速恢复原始model.py。
-
避坑 6：帧数未对齐"17k+5"网格，导致生成失败或截断
-
症状 ：工作流执行到
MiniMaxH3ImageToVideo节点时报错Invalid length，或生成的视频实际时长与设定时长不符（如设定 15 秒只生成了 12 秒）。 -
根本原因 ：MiniMax-H3 的 DiT 架构以 17 帧为一个基础块 （17k+5 网格），且帧率固定为 24fps。若输入的帧数不满足
(N - 5) % 17 == 0，模型会拒绝执行或静默截断至最近的有效值。 -
正确做法 ：必须使用本文工作流中的
ComfyMathExpression节点，表达式为：text
max(5, round(a * 24)) + (5 - (max(5, round(a * 24)) % 17)) % 17其中
a为用户输入的秒数。此公式会自动向上取整对齐网格。例如 15 秒 → 360 帧 → 对齐后仍为 360 帧（因(360-5)%17=0）；若输入非标时长，表达式会自动补齐。
避坑 7：漏设 Flash Attention 环境变量，性能回退至 SDPA
-
症状 ：已按步骤安装 SageAttention 并修改
attention.py，但生成速度与预期严重不符（标准画质 15 秒需 20+ 分钟），且启动日志中未见Flash Attention backend相关提示。 -
根本原因 ：ComfyUI 读取环境变量
COMFY_FLASH_ATTN_BACKEND来决定使用的注意力后端。若未设置，默认行为可能回退至 PyTorch 原生 SDPA（math或flash_sdp低效模式），且不会报错。 -
正确做法 ：启动脚本中强制显式声明以下两个环境变量：
bash
export COMFY_FLASH_ATTN_BACKEND=native export COMFY_FLASH_ATTN_REPACK_QKV=1启动后检查日志，确保出现
Flash Attention backend: native和Native Flash Attention QKV repack: enabled。
避坑 8：多实例并行时，端口与 SQLite 数据库锁冲突
-
症状 ：启动第 2 个 ComfyUI 实例时，报错
sqlite3.OperationalError: database is locked，或浏览器无法访问指定端口。 -
根本原因 ：多个 ComfyUI 实例默认使用同一个
user/comfyui.db数据库文件，SQLite 不支持多进程并发写入。同时，若多个实例绑定同一端口，会导致端口占用冲突。 -
正确做法：
-
为每个实例分配唯一端口（如 7681~7688）。
-
为每个实例分配独立数据库路径 （如
--database-url "sqlite:///./user/comfyui_${RANK}.db"）。 -
脚本中按
HIP_VISIBLE_DEVICES的序号（0~7）动态生成端口和 DB 路径，实现完全隔离。
-
避坑 9：长视频生成中后期 OOM，但显存尚有剩余
-
症状 ：采样进行到 60%~80% 时，ComfyUI 报
RuntimeError: HIP out of memory，但通过rocm-smi观察到显存并未完全占满（如占用 50GB/64GB）。 -
根本原因：PyTorch 的显存分配器在长时间运行中会产生大量碎片（Segmentation Fragmentation），导致即使总剩余显存足够，也无法分配连续的连续内存块来容纳中间激活张量。
-
正确做法：启动脚本中必须包含：
bash
PYTORCH_ALLOC_CONF=expandable_segments:True该参数允许 PyTorch 扩展已有显存段，大幅降低碎片率。本文的启动脚本已包含该配置，切勿删减。
避坑 10：LoRA 强度设置过高导致画面崩坏
-
症状 ：加载
minimax_h3_turbo_v4LoRA 后，生成视频出现严重颜色畸变、内容混乱或主体消失。 -
根本原因 ：蒸馏 LoRA 的权重是针对 4 步采样（CFG=1.0）优化设计的。若强行提高
strength_model（如 >1.2），或采样步数 >10 步，LoRA 特征会过度激活，破坏原模型的潜在空间分布。 -
正确做法：
-
LoraLoaderModelOnly节点中的strength_model严格保持为 1.0。 -
采样步数设置为 8 步 （本文
BasicScheduler已配置）。 -
务必 配合
res_multistep采样器和simple调度器使用，不要混用其他组合（如 DPM++ 或 Karras），否则蒸馏效果失效。 -
如果画面仍有瑕疵，优先调整
TESpeedMiniMaxH3中的processing_control_value（推荐 0.10~0.15），而非提高 LoRA 强度。
-
避坑总结速查表
| 序号 | 核心问题 | 一句话解决方案 | 错误代价 |
|---|---|---|---|
| 1 | DCU 不可见 | 挂载 /dev/kfd、/dev/dri、/dev/mkfd |
无法启动 |
| 2 | PyTorch 被覆盖 | 删除 requirements.txt 中 torch 行 | 内核报错 |
| 3 | SageAttention 崩溃 | 锁定版本 ==1.0.6 |
越界访问 |
| 4 | VM Fault 页错误 | attention.py 硬锁 BLOCK_M=64 | 内核崩溃 |
| 5 | TE-Speed 失效 | 升级 ComfyUI 至最新 + TE-Speed 3.2+ | 画面残影 |
| 6 | 帧数无效 | 套用 17k+5 对齐公式 | 生成截断 |
| 7 | 性能未达预期 | 强制设置 FLASH_ATTN_BACKEND=native | 回退 SDPA |
| 8 | 多实例锁冲突 | 独立端口 + 独立 DB 路径 | 启动失败 |
| 9 | 长序列 OOM | 设置 expandable_segments:True |
显存溢出 |
| 10 | LoRA 画面崩坏 | strength=1.0 + 步数=8 | 画面畸变 |
这十点避坑指南，几乎覆盖了从容器环境 → 依赖安装 → 底层算子 → 上层插件 → 系统并行的全部生命周期。希望读者能将此章节作为"应急预案"，遇到异常时优先对照排查。
十、总结
本文完整记录了在海光K100_AI单卡（64GB显存）上部署和优化ComfyUI+MiniMax-H3全流程视频生成方案的技术实践。从环境搭建、依赖适配、底层算子改造到上层加速插件调优，形成了一套可复制、可落地的国产算力AIGC工程方案。
核心成果
| 指标 | 优化前 | 优化后 |
|---|---|---|
| 单卡15秒高清视频（768×1376） | 无法稳定运行（OOM/崩溃） | ~22分钟 |
| 单卡15秒标准视频（0.5M像素） | ~一个半小时（回退SDPA） | ~7分钟（13倍提升） |
| 8卡并行吞吐 | 不适用 | 20分钟/120秒高清视频（8×线性扩展） |
技术贡献回顾
本方案的核心贡献可归纳为五个层次：
-
环境层：基于DTK 2604定制镜像 + DAS优化库，构建海光DCU专属的ComfyUI运行环境。
-
模型层：INT8量化主模型 + 4步蒸馏Turbo LoRA，将采样步数从20+步降至4~8步，降低推理延迟70%以上。
-
算子层 ：SageAttention 1.0.6算子融合加速（约11%提升）+
attention.py底层改造。后者是本方案的核心技术壁垒------通过Triton API兼容层修补、BLOCK_M/BLOCK_N=64硬锁定（规避VM Fault）、QKV重排优化，使Flash Attention在海光DCU上从"一碰就碎"变得"坚韧高效"。 -
调度层：TE-Speed-MiniMaxH3-OSS块级残差缓存（约45%提速），利用扩散模型相邻步隐空间变化平滑的数学本质，在不损失画质的前提下大幅减少计算量。
-
系统层：8实例并行部署，每卡独立端口与数据库，实现线性吞吐扩展。
实践意义
本方案的价值不仅在于技术指标的提升，更在于验证了国产算力平台（海光K100_AI）配合开源模型（MiniMax-H3）和社区工具（ComfyUI + TE-Speed + SageAttention）能够形成高效的AIGC生产闭环。
在信创背景下，这一实践为以下场景提供了可复用的技术范本：
-
国产DCU集群上的视频批量生成服务
-
基于ComfyUI的信创AIGC平台建设
-
大模型推理在非NVIDIA平台上的性能调优方法论
最终建议
对于计划复现本方案的读者，以下三条建议最为关键：
-
版本锁定是隐形的性能护城河：DTK 2604 + SageAttention 1.0.6 + flash-attn 2.8.x + TE-Speed 3.2+ 这一组合是经过大量试错后稳定的"配方"，任何单一版本变动都可能引入不可预知的问题。
-
国产硬件适配优先于算法优化 ：在K100_AI上，让Flash Attention"能跑"比"跑得快"更重要。
attention.py的修改牺牲了部分理论峰值性能，却换来了长序列推理的100%执行确定性。 -
避坑指南是应急预案而非可选项：本文第十章（踩坑避坑指南）记录的所有问题均来自实际调试过程中的真实故障，建议读者在部署前逐条对照检查，而非遇到问题后再回头查阅。
写在最后：国产算力平台的生态完善尚在途中，每一步优化都意味着与硬件手册、社区文档、源代码的三重对话。本文的每一条参数、每一处修改，背后都是数小时的调试与验证。希望这份"踩坑记录"能帮助后来者少走弯路，让更多AI应用能在国产DCU上高效运行。如果本文对你有帮助，欢迎点赞、收藏、评论交流。