llama-7B、vicuna-7b-delta-v1.1和vicuna-7b-v1.3——使用体验

Chatgpt的出现给NLP领域带来了让人振奋的消息,可以很逼真的模拟人的对话,回答人们提出的问题,不过Chatgpt参数量,规模,训练代价都很昂贵。

幸运的是,出现了开源的一些相对小的模型,可以在本地或者云端部署体验,

动手体验了下Vicuna-7b,翻译过来是小羊驼」(骆马),拥有70亿参数,据作者实验能达到GPT-4的90%性能。在作者官网发布了三个版本,其中3个月前发布了v1.1,17天前发布了v1.3。

官网:lmsys (Large Model Systems Organization)

环境:

ubuntu18.04

九天毕昇8核32G内存,仅有cpu

时间:2023年7月5号

llama-7B

python3 -m cli --model-path /root/transformer_model_7b --device cpu

文件:cli是fastchat/serve目录下的cli.py文件

cli.py内容

"""

Chat with a model with command line interface.

Usage:

python3 -m fastchat.serve.cli --model lmsys/vicuna-7b-v1.3

python3 -m fastchat.serve.cli --model lmsys/fastchat-t5-3b-v1.0

Other commands:

  • Type "!!exit" or an empty line to exit.

  • Type "!!reset" to start a new conversation.

"""

import argparse

import os

import re

import sys

from prompt_toolkit import PromptSession

from prompt_toolkit.auto_suggest import AutoSuggestFromHistory

from prompt_toolkit.completion import WordCompleter

from prompt_toolkit.history import InMemoryHistory

from prompt_toolkit.key_binding import KeyBindings

from rich.console import Console

from rich.live import Live

from rich.markdown import Markdown

from fastchat.model.model_adapter import add_model_args

from fastchat.modules.gptq import GptqConfig

from fastchat.serve.inference import ChatIO, chat_loop

class SimpleChatIO(ChatIO):

def prompt_for_input(self, role) -> str:

return input(f"{role}: ")

参数: --model-path 是指模型权重存放路径,这个目录下包含tokenizer.model文件,请仔细核对

参数: --device cpu 指定在cpu上运行

开始推理: 输入 who is Musk? what is baidu? china

要等待几分钟,看到几个词几个词的蹦出来

看回答,字数还不少,内容只能说是相关

看下cpu使用率

top

可以看到python3进程,推理时cpu使用率在81.1%,一直在波动。

换用v100有时候会到98%,甚至更高,速度会快很多,一次输出一小段文字

vicuna-7b-delta-v1.1

python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu

加载模型,中间过程要几分钟

python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [02:58<00:00, 89.16s/it]

开始推理:实现一个python斐波那契函数,可以看到能够写出函数,还简单介绍了怎么用

USER: hello

ASSISTANT: Hello! How can I help you today? Is there something you would like to know or talk about? I'm here to help answer any questions you may have.

USER: implement a python function to compute fibonacci numbers

ASSISTANT: Sure! Here's an implementation of a Python function that computes the Fibonacci sequence:

```python

def fibonacci(n):

if n <= 1:

return n

else:

return fibonacci(n-1) + fibonacci(n-2)

```

This function takes an integer `n` as input and returns the `n`-th Fibonacci number. The first two numbers in the sequence are 0 and 1, so the function handles these cases separately: if `n` is less than or equal to 1, it returns `n`, and otherwise it computes the sum of the previous two Fibonacci numbers and returns that.

To use the function, you can call it with an integer `n` and it will return the `n`-th Fibonacci number. For example:

```python

>>> fibonacci(5)

13

```

This would compute the 5-th Fibonacci number, which is 13.

USER: china

ASSISTANT: China is a country located in East Asia, bordered by several other countries, including Russia, Mongolia, North Korea, Vietnam, Laos, Myanmar (Burma), Bhutan, India, Nepal, and Pakistan. It is the world's most populous country, with over 1.4 billion people.

China has a long and complex history, dating back over 5,000 years. It is known for its rich cultural heritage, including its cuisine, literature, and arts. The country has a diverse range of landscapes, from the high plateaus in the west to the fertile river valleys in the east, and from the tropical beaches in the south to the frigid deserts in the north.

vicuna-7b-v1.3

python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu

参数:-m 后面指定cli.py文件,注意【.py】省略。python3 -m cli是python3执行cli.py文件,具体内容,要看cli.py写了什么

加载模型,等待几分钟

python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [03:17<00:00, 98.73s/it]

推理:

USER: Introduce China

ASSISTANT: China, officially known as the People's Republic of China (PRC), is a sovereign state located in East Asia. It is the world's most populous country, with a population of over 1.4 billion people, and the third-largest country by land area, covering approximately 9.6 million square kilometers.

China has a long and complex history, dating back more than 5,000 years. It has been ruled by various dynasties, including the Zhou, Qin, Han, Tang, Song, Yuan, Ming, and Qing dynasties, among others.

The People's Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the People's Republic of China, which was adopted in 1982. The government is divided into various branches, including the National People's Congress (NPC), the State Council, the Supreme People's Court, and the Central Military Commission.

China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.

The People's Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the People's Republic of China, which was adopted in 1982. The government is divided into various branches, including the National People's Congress (NPC), the State Council, the Supreme People's Court, and the Central Military Commission.

China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.

The economy of China has experienced rapid growth in recent decades, making it the world's second-largest economy by nominal GDP and the largest by purchasing power parity (PPP). The country is a leading global exporter of goods and services, with industries such as electronics, textiles, automobiles, and telecommunications playing a significant role in its economic development.

China has been actively involved in international affairs, participating as a permanent member of the United Nations Security Council and a member of various multilateral organizations, including the World Trade Organization (WTO), the Asia-Pacific Economic Cooperation (APEC), and the Group of 20 (G20) major economies.

USER: china

ASSISTANT: China is a country located in East Asia, covering approximately 9.6 million square kilometers. With a population of over 1.4 billion people, it is the most populous country in the world. The official language is Mandarin Chinese, and the capital city is Beijing. China has a rich history and a diverse cultural heritage, with many different ethnic groups and over 56 recognized ethnic minorities.

目前尚不能支持中文提问和中文回答

提问方式很重要:输入china,回答的只能说是相关,文字就少很多。输入 Introduce China,输出大段的介绍

相关推荐
韬小志20 小时前
【LLaMa-Factory】监督微调训练方法
人工智能·深度学习·llama
大拨鼠2 天前
【多模态读论文系列】LLaMA-Adapter V2论文笔记
论文阅读·人工智能·llama
努力的光头强3 天前
太炸裂了,Ollama跑本地模型已成为历史,现在都在使用这个工具,而且还能集成本地知识库
人工智能·ai·pdf·产品经理·llama
AIBigModel4 天前
LLaMA系列一直在假装开源...
开源·llama
三月七(爱看动漫的程序员)5 天前
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
人工智能·gpt·语言模型·自然语言处理·chatgpt·llama
励志成为美貌才华为一体的女子7 天前
基于LLaMA Factory对LLama 3指令微调的操作学习笔记
llama
HyperAI超神经8 天前
对标Hugging Face?GitHub Models新增OpenAI o1/Llama 3.2等, 新功能支持模型并排比较
人工智能·机器学习·github·llama·huggingface
努力的光头强10 天前
人工智能大模型赋能医疗健康产业白皮书(2023年)|附88页PDF文件下载
人工智能·算法·ai·pdf·产品经理·llama
cv2016_DL10 天前
CLIP改进
人工智能·深度学习·机器学习·计算机视觉·llama
马武寨山的猴子11 天前
【LLaMA-Factory】【Windows】:在windows操作系统配置大模型微调框架LLaMA-Factory
人工智能·windows·机器学习·llama