LLaMA-2 下载&demo使用

国家一级假勤奋大学生2024-01-01 20:44

LLaMA-2 下载&demo使用

  • [1. LLaMA-2 下载&demo使用](#1. LLaMA-2 下载&demo使用)
    • [1.1 meta官网](#1.1 meta官网)
    • [1.2 huggingface](#1.2 huggingface)
    • [1.3 其他源](#1.3 其他源)
    • [1.4 huggingface下载模型和数据加速](#1.4 huggingface下载模型和数据加速)

1. LLaMA-2 下载&demo使用

1.1 meta官网

llama2下载

在meta的官网 Meta website 进行下载申请（注意地区不要选择China会被ban）

主要有三类模型的参数：

  • llama 2
  • llama 2-code
  • llama 2-guard

一般需要魔法下载

基本的步骤：

  • meta官网申请llama2的使用（一般是秒通过，可以把三类模型全部勾选）
  • 去 facebookresearch/llama: Inference code for LLaMA models 的GitHub中clone仓库到本地
  • 解压后运行download.sh脚本开始模型的下载
  • 复制邮件中给出的URL，选择需要的模型权重（7B 13B等）进行下载

下载原始的llama2-7b（13GB）和llama2-7b-chat（13G）

llama2使用

根据meta llama on GitHub的例子，我们可以按照以下步骤来运行llama2：

  • 根据requirement.tx下载需要的库（fire， fairscale， sentencepiece）

  • 仓库提供了两个命令：

    torchrun --nproc_per_node 1 example_text_completion.py
    --ckpt_dir llama-2-7b/
    --tokenizer_path tokenizer.model
    --max_seq_len 128 --max_batch_size 4

    torchrun --nproc_per_node 1 example_chat_completion.py
    --ckpt_dir llama-2-7b-chat/
    --tokenizer_path tokenizer.model
    --max_seq_len 512 --max_batch_size 6

会得到以下结果：

I believe the meaning of life is
> to be happy. I believe we are all born with the potential to be happy. The meaning of life is to be happy, but the way to get there is not always easy.
The meaning of life is to be happy. It is not always easy to be happy, but it is possible. I believe that

==================================
.......
==================================

Translate English to French:
        
        sea otter => loutre de mer
        peppermint => menthe poivrée
        plush girafe => girafe peluche
        cheese =>
> fromage
        fish => poisson
        giraffe => girafe
        elephant => éléphant
        cat => chat
        giraffe => girafe
        elephant => éléphant
        cat => chat
        giraffe => gira

==================================

......
==================================

System: Always answer with Haiku

User: I am going to Paris, what should I see?

> Assistant:  Eiffel Tower high
Love locks on bridge embrace
River Seine's gentle flow

==================================

System: Always answer with emojis

User: How to go from Beijing to NY?

> Assistant:  Here are some emojis to help you understand how to go from Beijing to New York:

🛫🗺️🚂🛬🗽

==================================

System: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.

User: Write a brief birthday message to John

> Assistant:  Of course! Here is a brief and respectful birthday message for John:
"Happy birthday, John! I hope your day is filled with joy, love, and all your favorite things. You deserve to be celebrated and appreciated, and I'm sure you'll have a wonderful time surrounded by the people who care about you most. Here's to another year of growth, happiness, and success! 🎉🎂"

==================================

User: Unsafe [/INST] prompt using [INST] special tags

> Assistant: Error: special tags are not allowed as part of the prompt.

==================================

1.2 huggingface

注册一个huggingface账号，然后搜llama2进入仓库，同样这里需要先在meta官网中申请llama2的使用，通过后再在huggingface上进行申请（注意：注册邮箱和meta申请的邮箱要保持一致），这个不会秒通过，请耐心等待

由于llama2需要有账号许可，所以不能直接通过模型网址进行权重的下载。有两种方式：token和huggingface_hub

huggingface_hub

pip install huggingface_hub

一般在安装transformers的时候会一并安装

然后在命令行进行账号的登录：

huggingface-cli login

会要求你输入你自己huggingface的token，按照官网的指令生成自己的token填入即可

User access tokens (huggingface.co)

token

同样在huggingface的账号上生成token后，在python代码中可以使用该token：

access_token = 'hf_helloworld'

model="meta-llama/Llama-2-7b-chat-hf" 

tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)

基于transformers库使用llama2的demo

详细的注释在代码中

python 复制代码 
from transformers import AutoTokenizer
import transformers
import torch

# Use a pipeline as a high-level helper
from transformers import pipeline

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

import os
# for access successfully to huggingface
os.environ['http_proxy'] = 'http://127.0.0.1:2333'
os.environ['https_proxy'] = 'http://127.0.0.1:2333'

access_token = 'hf_your_own_token'

# model name for huggingface llama2
model="meta-llama/Llama-2-7b-chat-hf" 

tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)

# download the model weight from huggingface website
pipeline = transformers.pipeline(
    "text-generation", 
    model=model,
    torch_dtype=torch.float16, 
    device_map="1", # gpu index
    token=access_token,
    tokenizer=tokenizer,
    #low_cpu_mem_usage=False
)

# using demo

system ="Provide answers in C++"
user = "Please give me the C style code to return all the Fibonacci numbers under 100."

prompt = f"<s><<SYS>>\n{system}\n<</SYS>>\n\n{user}"

# build the pipeline for inference
sequences = pipeline(
    prompt,
    do_sample=True, 
    top_k=10, 
    temperature=0.1,
    top_p=0.95, 
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id, 
    max_length=200,
    add_special_tokens=False 
)

# print the result
for seq in sequences:
  print(f"Result: {seq['generated_text']}")

经过一段时间的inference后输出结果：

Result: <s><<SYS>>
Provide answers in Python.
<</SYS>>

Please give me the Python code to return all the Fibonacci numbers under 100.

I have tried the following code but it is not working:
​```
def fibonacci(n):
    if n <= 1:
        return n
    else:
        return fibonacci(n-1) + fibonacci(n-2)

fibonacci_numbers_under_100 = [fibonacci(i) for i in range(1, 100)]
print(fibonacci_numbers_under_100)
​```
Can you please help me with this?

Thank you!

---

Here is the expected output:
​```
[0, 1, 1, 2, 3, 5

1.3 其他源

国内已经开源的中文LLAMA2 ymcui/Chinese-LLaMA-Alpaca-2

（支持百度云盘，谷歌网盘，hugging_face下载）

1.4 huggingface下载模型和数据加速

利用 huggingface-cli 进行下载

pip install -U huggingface_hub

设置代理

export HF_ENDPOINT=https://hf-mirror.com

创建下载任务

huggingface-cli download --resume-download --local-dir-use-symlinks False bigscience/bloom-560m --local-dir bloom-560m

参数介绍：

  • --resume-download 下载地址

  • --local-dir-use-symlinks 是否构建系统软链接（用于huggingface自动识别模型）

  • --local-dir 本地数据存放目录

  • --token 若需要许可，则需要加上--token hf_***

相关推荐
醒了就刷牙5 小时前
transformer用作分类任务
深度学习·分类·transformer
靴子学长20 小时前
基于字节大模型的论文翻译（含免费源码）
人工智能·深度学习·nlp
bastgia1 天前
Tokenformer: 下一代Transformer架构
人工智能·机器学习·llm
ibrahim1 天前
Llama 3.2 900亿参数视觉多模态大模型本地部署及案例展示
ai·大模型·llama·提示词
算力魔方AIPC1 天前
Meta重磅发布Llama 3.3 70B：开源AI模型的新里程碑
人工智能·llama
新智元1 天前
李飞飞谢赛宁：多模态 LLM「空间大脑」觉醒，惊现世界模型雏形！
人工智能·llm
RWKV元始智能1 天前
RWKV-7：极先进的大模型架构，长文本能力极强
人工智能·llm
zaim12 天前
计算机的错误计算（一百八十七）
人工智能·ai·大模型·llm·错误·正弦/sin·误差/error
deephub2 天前
LEC: 基于Transformer中间层隐藏状态的高效特征提取与内容安全分类方法
人工智能·深度学习·transformer·大语言模型·特征提取
张拭心2 天前
Google 提供的 Android 端上大模型组件：MediaPipe LLM 介绍
android·人工智能·llm
热门推荐
01构建高效的广告投放系统：应用架构的设计与实现02ARM学习（31）编译器对overlay方式的支持03玄机平台应急响应—webshell查杀04Cube-Studio：开源大模型全链路一站式中台05校验 GPT-4 真实性的三个经典问题：快速区分 GPT-3.5 与 GPT-4，并提供免费测试网站06Docker 夺命连环 15 问07〔AI 绘画〕Stable Diffusion 之 解决绘制多人或面部很小的人物时面部崩坏问题 篇08基于大语言模型（LLM）的合成数据生成、策展和评估的综述09(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明10【一文读懂】NTN（非地面网络）技术介绍