llama3入门训练和部署

教程一

https://www.youtube.com/watch?v=oxTVzGwKeoU

https://www.youtube.com/watch?v=oxTVzGwKeoU

  1. 下载 alpaca 数据集(斯坦福大学的数据集,通过gpt 转换成)
    https://huggingface.co/datasets/shibing624/alpaca-zh
  2. 将自己的数据集json 添加到上面下载的数据集中
  3. 使用 unsloth 进行微调 python app.py
    训练后的模型训练为 gguf 格式
python 复制代码
from unsloth import FastLanguageModel
import torch

from trl import SFTTrainer
from transformers import TrainingArguments




max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

# 4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [
    "unsloth/mistral-7b-bnb-4bit",
    "unsloth/mistral-7b-instruct-v0.2-bnb-4bit",
    "unsloth/llama-2-7b-bnb-4bit",
    "unsloth/gemma-7b-bnb-4bit",
    "unsloth/gemma-7b-it-bnb-4bit", # Instruct version of Gemma 7b
    "unsloth/gemma-2b-bnb-4bit",
    "unsloth/gemma-2b-it-bnb-4bit", # Instruct version of Gemma 2b
    "unsloth/llama-3-8b-bnb-4bit", # [NEW] 15 Trillion token Llama-3
] # More models at https://huggingface.co/unsloth

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3-8b-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)

model = FastLanguageModel.get_peft_model(
    model,
    r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 16,
    lora_dropout = 0, # Supports any, but = 0 is optimized
    bias = "none",    # Supports any, but = "none" is optimized
    # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = False,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
)

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN
def formatting_prompts_func(examples):
    instructions = examples["instruction"]
    inputs       = examples["input"]
    outputs      = examples["output"]
    texts = []
    for instruction, input, output in zip(instructions, inputs, outputs):
        # Must add EOS_TOKEN, otherwise your generation will go on forever!
        text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN
        texts.append(text)
    return { "text" : texts, }
pass

from datasets import load_dataset

file_path = "/home/Ubuntu/alpaca_gpt4_data_zh.json"


dataset = load_dataset("json", data_files={"train": file_path}, split="train")

dataset = dataset.map(formatting_prompts_func, batched = True,)




trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        max_steps = 60,
        learning_rate = 2e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
    ),
)

trainer_stats = trainer.train()

model.save_pretrained_gguf("dir", tokenizer, quantization_method = "q4_k_m")
model.save_pretrained_gguf("dir", tokenizer, quantization_method = "q8_0")
model.save_pretrained_gguf("dir", tokenizer, quantization_method = "f16") 
  1. 本地加载微调好的大模型
    使用工具: Ollama / LM Studio
    先安装 ollama
    创建一个空文件,Modelfile ,输入:

使用:

在这里插入图片描述

教程二

https://www.bilibili.com/video/BV1AH4y137tR/

1.准备数据 json 格式 数据集文件可以是多个json 文件

2.官网下载模型已微调的中文模型

https://huggingface.co/shenzhi-wang/Llama3-8B-Chinese-Chat-GGUF-8bit/tree/v1

  1. 微调工具: peft / llamafactory /unsloth

  2. 微调方式: lora 微调之后并不是模型的所有,需要再合并上原始的模型 (官网有脚本案例)

  3. 量化 llama.cpp (github) 注意: 进入llama.cpp 文件的格式都必须是.gguf 的格式,有工具可以进行转换

1)格式转化

  1. quantize.exe
  1. 部署 ollama lmstudio

ollama: modefile文件里修改模型路径

  1. phidata 做本地知识库
相关推荐
Allen_LVyingbo20 分钟前
数智读书笔记系列035《未来医疗:医疗4.0引领第四次医疗产业变革》
人工智能·经验分享·笔记·健康医疗
zzc92125 分钟前
时频图数据集更正程序,去除坐标轴白边及调整对应的标签值
人工智能·深度学习·数据集·标签·时频图·更正·白边
isNotNullX27 分钟前
什么是数据分析?常见方法全解析
大数据·数据库·数据仓库·人工智能·数据分析
riveting36 分钟前
明远智睿H618:开启多场景智慧生活新时代
人工智能·嵌入式硬件·智能硬件·lga封装·3506
夜阑卧听风吹雨,铁马冰河入梦来1 小时前
Spring AI 阿里巴巴学习
人工智能·学习·spring
c7691 小时前
【文献笔记】Automatic Chain of Thought Prompting in Large Language Models
人工智能·笔记·语言模型·论文笔记
Blossom.1182 小时前
机器学习在智能供应链中的应用:需求预测与物流优化
人工智能·深度学习·神经网络·机器学习·计算机视觉·机器人·语音识别
Gyoku Mint2 小时前
深度学习×第4卷:Pytorch实战——她第一次用张量去拟合你的轨迹
人工智能·pytorch·python·深度学习·神经网络·算法·聚类
zzywxc7872 小时前
AI大模型的技术演进、流程重构、行业影响三个维度的系统性分析
人工智能·重构
点控云2 小时前
智能私域运营中枢:从客户视角看 SCRM 的体验革新与价值重构
大数据·人工智能·科技·重构·外呼系统·呼叫中心