Deepspeed : AttributeError: ‘DummyOptim‘ object has no attribute ‘step‘

题意 :尝试在一个名为 DummyOptim 的对象上调用 .step() 方法,但是这个对象并没有定义这个方法

问题背景:

I want to use deepspeed for training LLMs along with Huggingface Trainer. But when I use deepspeed along with trainer I get error "AttributeError: 'DummyOptim' object has no attribute 'step'". Below is my code

尝试结合使用 DeepSpeed 和 Hugging Face 的 Trainer API 来训练大型语言模型(LLMs)时遇到 "AttributeError: 'DummyOptim' object has no attribute 'step'" 这个错误,下面是我的代码:

python 复制代码
import argparse
import numpy as np
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM

from trl import DPOTrainer, DPOConfig
def preprocess_data(item):
    return {
        'prompt': 'Instruct: ' + item['prompt'] + '\n',
        'chosen': 'Output: ' + item['chosen'],
        'rejected': 'Output: ' + item['rejected']
    }        

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--epochs", type=int, default=1)
    parser.add_argument("--beta", type=float, default=0.1)
    parser.add_argument("--batch_size", type=int, default=4)
    parser.add_argument("--lr", type=float, default=1e-6)
    parser.add_argument("--seed", type=int, default=2003)
    parser.add_argument("--model_name", type=str, default="EleutherAI/pythia-14m")
    parser.add_argument("--dataset_name", type=str, default="jondurbin/truthy-dpo-v0.1")
    parser.add_argument("--local_rank", type=int, default=0)

    args = parser.parse_args()

    # Determine device based on local_rank
    device = torch.device("cuda", args.local_rank) if torch.cuda.is_available() else torch.device("cpu")


    tokenizer = AutoTokenizer.from_pretrained(args.model_name)
    tokenizer.pad_token = tokenizer.eos_token
    model = AutoModelForCausalLM.from_pretrained(args.model_name).to(device)
    ref_model = AutoModelForCausalLM.from_pretrained(args.model_name).to(device)

    dataset = load_dataset(args.dataset_name, split="train")
    dataset = dataset.map(preprocess_data)

    # Split the dataset into training and validation sets
    dataset = dataset.train_test_split(test_size=0.1, seed=args.seed)
    train_dataset = dataset['train']
    val_dataset = dataset['test']

    training_args = DPOConfig(
        learning_rate=args.lr,
        num_train_epochs=args.epochs,
        per_device_train_batch_size=args.batch_size,
        logging_steps=10,
        remove_unused_columns=False,
        max_length=1024,
        max_prompt_length=512,
        deepspeed="ds_config.json"       
    )

    

    # Verify and print embedding dimensions before finetuning
    print("Base model embedding dimension:", model.config.hidden_size)

    model.train()
    ref_model.eval()

    dpo_trainer = DPOTrainer(
        model,
        ref_model,
        beta=args.beta,
        train_dataset=train_dataset,
        eval_dataset=val_dataset,
        tokenizer=tokenizer,
        args=training_args,
    )

    dpo_trainer.train()
    # Evaluate
    evaluation_results = dpo_trainer.evaluate()
    print("Evaluation Results:", evaluation_results)

    save_model_name = 'finetuned_model'
    model.save_pretrained(save_model_name)

if __name__ == "__main__":
    main()

The config file used is the below one 使用的配置文件是下面的这个:

python 复制代码
{
"zero_optimization": {
        "stage": 3,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": true
        },
        "offload_param": {
            "device": "cpu",
            "pin_memory": true
        },
        "overlap_comm": true,
        "contiguous_gradients": true,
        "sub_group_size": 1e9,
        "reduce_bucket_size": "auto",
        "stage3_prefetch_bucket_size": "auto",
        "stage3_param_persistence_threshold": "auto",
        "stage3_max_live_parameters": 1e9,
        "stage3_max_reuse_distance": 1e9,
        "stage3_gather_16bit_weights_on_model_save": true
    },
"bf16": {
    "enabled": "auto"
},
"fp16": {
    "enabled": "auto",
    "loss_scale": 0,
    "initial_scale_power": 32,
    "loss_scale_window": 1000,
    "hysteresis": 2,
    "min_loss_scale": 1
},

"gradient_accumulation_steps": "auto",
"gradient_clipping": "auto",
"train_batch_size": "auto",
"train_micro_batch_size_per_gpu": "auto",
"wall_clock_breakdown": false,
"flops_profiler": {
    "enabled": false,
    "detailed": false
},
"optimizer": {
    "type": "Lamb",
    "params": {
    "lr": "auto",
    "betas": [0.9, 0.999],
    "eps": "auto",
    "weight_decay": "auto"
    }
},
"zero_allow_untested_optimizer": true
}

The code works with out deepspeed. I have torch=2.3.1, deepspeed =0.14.5, trl=0.9.4 and CUDA Version: 12.5.

在没有使用 DeepSpeed 的情况下,代码可以正常工作。当前的软件版本配置为:PyTorch 2.3.1,DeepSpeed 0.14.5,TRL 0.9.4,以及 CUDA 版本 12.5。

Appreciate any hint on this ! 非常感谢您在这方面的任何提示!

问题解决:

python 复制代码
from accelerate.utils import DistributedType

training_args.distributed_state.distributed_type = DistributedType.DEEPSPEED

adding this solves the issue 添加这个解决了问题

相关推荐
进击的六角龙20 分钟前
Python中处理Excel的基本概念(如工作簿、工作表等)
开发语言·python·excel
一只爱好编程的程序猿40 分钟前
Java后台生成指定路径下创建指定名称的文件
java·python·数据下载
Aniay_ivy1 小时前
深入探索 Java 8 Stream 流:高效操作与应用场景
java·开发语言·python
gonghw4031 小时前
DearPyGui学习
python·gui
向阳12181 小时前
Bert快速入门
人工智能·python·自然语言处理·bert
engchina1 小时前
Neo4j 和 Python 初学者指南:如何使用可选关系匹配优化 Cypher 查询
数据库·python·neo4j
兆。1 小时前
掌握 PyQt5:从零开始的桌面应用开发
开发语言·爬虫·python·qt
南宫理的日知录2 小时前
99、Python并发编程:多线程的问题、临界资源以及同步机制
开发语言·python·学习·编程学习
coberup2 小时前
django Forbidden (403)错误解决方法
python·django·403错误
龙哥说跨境2 小时前
如何利用指纹浏览器爬虫绕过Cloudflare的防护?
服务器·网络·python·网络爬虫