LLaMA-2 下载&demo使用

国家一级假勤奋大学生2024-01-01 20:44

LLaMA-2 下载&demo使用

  • [1. LLaMA-2 下载&demo使用](#1. LLaMA-2 下载&demo使用)
    • [1.1 meta官网](#1.1 meta官网)
    • [1.2 huggingface](#1.2 huggingface)
    • [1.3 其他源](#1.3 其他源)
    • [1.4 huggingface下载模型和数据加速](#1.4 huggingface下载模型和数据加速)

1. LLaMA-2 下载&demo使用

1.1 meta官网

llama2下载

在meta的官网 Meta website 进行下载申请（注意地区不要选择China会被ban）

主要有三类模型的参数：

  • llama 2
  • llama 2-code
  • llama 2-guard

一般需要魔法下载

基本的步骤：

  • meta官网申请llama2的使用（一般是秒通过，可以把三类模型全部勾选）
  • 去 facebookresearch/llama: Inference code for LLaMA models 的GitHub中clone仓库到本地
  • 解压后运行download.sh脚本开始模型的下载
  • 复制邮件中给出的URL，选择需要的模型权重（7B 13B等）进行下载

下载原始的llama2-7b（13GB）和llama2-7b-chat（13G）

llama2使用

根据meta llama on GitHub的例子，我们可以按照以下步骤来运行llama2：

  • 根据requirement.tx下载需要的库（fire， fairscale， sentencepiece）

  • 仓库提供了两个命令：

    torchrun --nproc_per_node 1 example_text_completion.py
    --ckpt_dir llama-2-7b/
    --tokenizer_path tokenizer.model
    --max_seq_len 128 --max_batch_size 4

    torchrun --nproc_per_node 1 example_chat_completion.py
    --ckpt_dir llama-2-7b-chat/
    --tokenizer_path tokenizer.model
    --max_seq_len 512 --max_batch_size 6

会得到以下结果：

I believe the meaning of life is
> to be happy. I believe we are all born with the potential to be happy. The meaning of life is to be happy, but the way to get there is not always easy.
The meaning of life is to be happy. It is not always easy to be happy, but it is possible. I believe that

==================================
.......
==================================

Translate English to French:
        
        sea otter => loutre de mer
        peppermint => menthe poivrée
        plush girafe => girafe peluche
        cheese =>
> fromage
        fish => poisson
        giraffe => girafe
        elephant => éléphant
        cat => chat
        giraffe => girafe
        elephant => éléphant
        cat => chat
        giraffe => gira

==================================

......
==================================

System: Always answer with Haiku

User: I am going to Paris, what should I see?

> Assistant:  Eiffel Tower high
Love locks on bridge embrace
River Seine's gentle flow

==================================

System: Always answer with emojis

User: How to go from Beijing to NY?

> Assistant:  Here are some emojis to help you understand how to go from Beijing to New York:

🛫🗺️🚂🛬🗽

==================================

System: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.

User: Write a brief birthday message to John

> Assistant:  Of course! Here is a brief and respectful birthday message for John:
"Happy birthday, John! I hope your day is filled with joy, love, and all your favorite things. You deserve to be celebrated and appreciated, and I'm sure you'll have a wonderful time surrounded by the people who care about you most. Here's to another year of growth, happiness, and success! 🎉🎂"

==================================

User: Unsafe [/INST] prompt using [INST] special tags

> Assistant: Error: special tags are not allowed as part of the prompt.

==================================

1.2 huggingface

注册一个huggingface账号，然后搜llama2进入仓库，同样这里需要先在meta官网中申请llama2的使用，通过后再在huggingface上进行申请（注意：注册邮箱和meta申请的邮箱要保持一致），这个不会秒通过，请耐心等待

由于llama2需要有账号许可，所以不能直接通过模型网址进行权重的下载。有两种方式：token和huggingface_hub

huggingface_hub

pip install huggingface_hub

一般在安装transformers的时候会一并安装

然后在命令行进行账号的登录：

huggingface-cli login

会要求你输入你自己huggingface的token，按照官网的指令生成自己的token填入即可

User access tokens (huggingface.co)

token

同样在huggingface的账号上生成token后，在python代码中可以使用该token：

access_token = 'hf_helloworld'

model="meta-llama/Llama-2-7b-chat-hf" 

tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)

基于transformers库使用llama2的demo

详细的注释在代码中

python 复制代码 
from transformers import AutoTokenizer
import transformers
import torch

# Use a pipeline as a high-level helper
from transformers import pipeline

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

import os
# for access successfully to huggingface
os.environ['http_proxy'] = 'http://127.0.0.1:2333'
os.environ['https_proxy'] = 'http://127.0.0.1:2333'

access_token = 'hf_your_own_token'

# model name for huggingface llama2
model="meta-llama/Llama-2-7b-chat-hf" 

tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)

# download the model weight from huggingface website
pipeline = transformers.pipeline(
    "text-generation", 
    model=model,
    torch_dtype=torch.float16, 
    device_map="1", # gpu index
    token=access_token,
    tokenizer=tokenizer,
    #low_cpu_mem_usage=False
)

# using demo

system ="Provide answers in C++"
user = "Please give me the C style code to return all the Fibonacci numbers under 100."

prompt = f"<s><<SYS>>\n{system}\n<</SYS>>\n\n{user}"

# build the pipeline for inference
sequences = pipeline(
    prompt,
    do_sample=True, 
    top_k=10, 
    temperature=0.1,
    top_p=0.95, 
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id, 
    max_length=200,
    add_special_tokens=False 
)

# print the result
for seq in sequences:
  print(f"Result: {seq['generated_text']}")

经过一段时间的inference后输出结果：

Result: <s><<SYS>>
Provide answers in Python.
<</SYS>>

Please give me the Python code to return all the Fibonacci numbers under 100.

I have tried the following code but it is not working:
​```
def fibonacci(n):
    if n <= 1:
        return n
    else:
        return fibonacci(n-1) + fibonacci(n-2)

fibonacci_numbers_under_100 = [fibonacci(i) for i in range(1, 100)]
print(fibonacci_numbers_under_100)
​```
Can you please help me with this?

Thank you!

---

Here is the expected output:
​```
[0, 1, 1, 2, 3, 5

1.3 其他源

国内已经开源的中文LLAMA2 ymcui/Chinese-LLaMA-Alpaca-2

（支持百度云盘，谷歌网盘，hugging_face下载）

1.4 huggingface下载模型和数据加速

利用 huggingface-cli 进行下载

pip install -U huggingface_hub

设置代理

export HF_ENDPOINT=https://hf-mirror.com

创建下载任务

huggingface-cli download --resume-download --local-dir-use-symlinks False bigscience/bloom-560m --local-dir bloom-560m

参数介绍：

  • --resume-download 下载地址

  • --local-dir-use-symlinks 是否构建系统软链接（用于huggingface自动识别模型）

  • --local-dir 本地数据存放目录

  • --token 若需要许可，则需要加上--token hf_***

相关推荐
陌上阳光5 小时前
动手学深度学习68 Transformer
人工智能·深度学习·transformer
Power202466615 小时前
NLP论文速读（NeurIPS 2024）|大语言模型在评估的时候更倾向于自己生成的内容
人工智能·深度学习·机器学习·计算机视觉·语言模型·自然语言处理·nlp
deephub17 小时前
TSMamba：基于Mamba架构的高效时间序列预测基础模型
人工智能·深度学习·transformer·时间序列·mamba
Lor :)17 小时前
深入浅出 ChatGPT 底层原理：Transformer
深度学习·chatgpt·transformer
Jacob_AI19 小时前
P-tuning、Prompt-tuning和Prefix-tuning区别是什么？
语言模型·nlp·prompt·transformer
机器学习之心2 天前
Transformer-GRU、Transformer、CNN-GRU、GRU、CNN五模型多变量回归预测
多输入单输出回归预测·cnn·gru·transformer·cnn-gru
Nil_cxc2 天前
机器学习周报（transformer学习1）
学习·机器学习·transformer
字节跳动开源2 天前
最高提升20倍吞吐量！豆包大模型团队发布全新 RLHF 框架，现已开源！
开源·llm·强化学习
kida_yuan2 天前
【从零开始】6. RAG 应用性能压测工具（番外篇）
后端·llm·测试
真忒修斯之船2 天前
大模型分布式训练并行技术（四）张量并行
面试·llm·aigc
热门推荐
01聊聊 Python 中的同步原语，为什么有了 GIL 还需要同步原语02【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总03Windows10安装PCL1.14.0及点云配准04C++——超强级病毒代码05全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）06组基轨迹建模 GBTM的介绍与实现（Stata 或 R）07硬件预取之空间预取08Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO09Coze扣子平台完整体验和实践（附国内和国际版对比）10【SCS-CN】SCS-CN模型中CN值的确定