记录torch运行的bug

Traceback (most recent call last):

File "/mnt2/wsj/table/basetest/test_single.py", line 243, in <module>

QWen2VL()

File "/mnt2/wsj/table/basetest/test_single.py", line 116, in QWen2VL

generated_ids = model.generate(**inputs, max_new_tokens=512)

File "/home/turing1/miniconda3/envs/MiniCPMV_wsj/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context

return func(*args, **kwargs)

File "/home/turing1/miniconda3/envs/MiniCPMV_wsj/lib/python3.10/site-packages/transformers/generation/utils.py", line 2048, in generate

result = self._sample(

File "/home/turing1/miniconda3/envs/MiniCPMV_wsj/lib/python3.10/site-packages/transformers/generation/utils.py", line 3044, in _sample

next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)

RuntimeError: CUDA error: device-side assert triggered

Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

试试看只使用一个gpu

python 复制代码
    model = Qwen2VLForConditionalGeneration.from_pretrained(
        weight_path,
        torch_dtype=torch.bfloat16,
        device_map="sequential",
    )
    processor = AutoProcessor.from_pretrained(weight_path)
python 复制代码
    device = torch.device('cuda:0')    
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=True,
        return_tensors="pt",
    ).to(device)
    generated_ids = model.generate(**inputs, max_new_tokens=512)
    generated_ids_trimmed = [
        out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
    ]
    output_text = processor.batch_decode(
        generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
    )
相关推荐
毛飞龙4 小时前
Python类(class)参数self的理解
python··self
魔尔助理顾问4 小时前
系统整理Python的循环语句和常用方法
开发语言·后端·python
Ai墨芯1116 小时前
深度学习水论文:特征提取
人工智能·深度学习
SHIPKING3936 小时前
【机器学习&深度学习】LLamaFactory微调效果与vllm部署效果不一致如何解决
人工智能·深度学习·机器学习
颜颜yan_6 小时前
Python面向对象编程详解:从零开始掌握类的声明与使用
开发语言·redis·python
闻道且行之6 小时前
Windows|CUDA和cuDNN下载和安装,默认安装在C盘和不安装在C盘的两种方法
windows·深度学习·cuda·cudnn
我的ID配享太庙呀7 小时前
Django 科普介绍:从入门到了解其核心魅力
数据库·后端·python·mysql·django·sqlite
@蓝莓果粒茶8 小时前
LeetCode第350题_两个数组的交集II
c++·python·学习·算法·leetcode·职场和发展·c#
生医转码,四海为家8 小时前
零基础-动手学深度学习-6.6 卷积神经网络(LeNet)
人工智能·深度学习·cnn
FinAnalyzer8 小时前
如何在 InsCodeAI 上搭建并使用 Jupyter Notebook 环境?
ide·python·jupyter