llama-7B、vicuna-7b-delta-v1.1和vicuna-7b-v1.3——使用体验

Chatgpt的出现给NLP领域带来了让人振奋的消息,可以很逼真的模拟人的对话,回答人们提出的问题,不过Chatgpt参数量,规模,训练代价都很昂贵。

幸运的是,出现了开源的一些相对小的模型,可以在本地或者云端部署体验,

动手体验了下Vicuna-7b,翻译过来是小羊驼」(骆马),拥有70亿参数,据作者实验能达到GPT-4的90%性能。在作者官网发布了三个版本,其中3个月前发布了v1.1,17天前发布了v1.3。

官网:lmsys (Large Model Systems Organization)

环境:

ubuntu18.04

九天毕昇8核32G内存,仅有cpu

时间:2023年7月5号

llama-7B

python3 -m cli --model-path /root/transformer_model_7b --device cpu

文件:cli是fastchat/serve目录下的cli.py文件

cli.py内容

"""

Chat with a model with command line interface.

Usage:

python3 -m fastchat.serve.cli --model lmsys/vicuna-7b-v1.3

python3 -m fastchat.serve.cli --model lmsys/fastchat-t5-3b-v1.0

Other commands:

  • Type "!!exit" or an empty line to exit.

  • Type "!!reset" to start a new conversation.

"""

import argparse

import os

import re

import sys

from prompt_toolkit import PromptSession

from prompt_toolkit.auto_suggest import AutoSuggestFromHistory

from prompt_toolkit.completion import WordCompleter

from prompt_toolkit.history import InMemoryHistory

from prompt_toolkit.key_binding import KeyBindings

from rich.console import Console

from rich.live import Live

from rich.markdown import Markdown

from fastchat.model.model_adapter import add_model_args

from fastchat.modules.gptq import GptqConfig

from fastchat.serve.inference import ChatIO, chat_loop

class SimpleChatIO(ChatIO):

def prompt_for_input(self, role) -> str:

return input(f"{role}: ")

参数: --model-path 是指模型权重存放路径,这个目录下包含tokenizer.model文件,请仔细核对

参数: --device cpu 指定在cpu上运行

开始推理: 输入 who is Musk? what is baidu? china

要等待几分钟,看到几个词几个词的蹦出来

看回答,字数还不少,内容只能说是相关

看下cpu使用率

top

可以看到python3进程,推理时cpu使用率在81.1%,一直在波动。

换用v100有时候会到98%,甚至更高,速度会快很多,一次输出一小段文字

vicuna-7b-delta-v1.1

python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu

加载模型,中间过程要几分钟

python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [02:58<00:00, 89.16s/it]

开始推理:实现一个python斐波那契函数,可以看到能够写出函数,还简单介绍了怎么用

USER: hello

ASSISTANT: Hello! How can I help you today? Is there something you would like to know or talk about? I'm here to help answer any questions you may have.

USER: implement a python function to compute fibonacci numbers

ASSISTANT: Sure! Here's an implementation of a Python function that computes the Fibonacci sequence:

```python

def fibonacci(n):

if n <= 1:

return n

else:

return fibonacci(n-1) + fibonacci(n-2)

```

This function takes an integer `n` as input and returns the `n`-th Fibonacci number. The first two numbers in the sequence are 0 and 1, so the function handles these cases separately: if `n` is less than or equal to 1, it returns `n`, and otherwise it computes the sum of the previous two Fibonacci numbers and returns that.

To use the function, you can call it with an integer `n` and it will return the `n`-th Fibonacci number. For example:

```python

>>> fibonacci(5)

13

```

This would compute the 5-th Fibonacci number, which is 13.

USER: china

ASSISTANT: China is a country located in East Asia, bordered by several other countries, including Russia, Mongolia, North Korea, Vietnam, Laos, Myanmar (Burma), Bhutan, India, Nepal, and Pakistan. It is the world's most populous country, with over 1.4 billion people.

China has a long and complex history, dating back over 5,000 years. It is known for its rich cultural heritage, including its cuisine, literature, and arts. The country has a diverse range of landscapes, from the high plateaus in the west to the fertile river valleys in the east, and from the tropical beaches in the south to the frigid deserts in the north.

vicuna-7b-v1.3

python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu

参数:-m 后面指定cli.py文件,注意【.py】省略。python3 -m cli是python3执行cli.py文件,具体内容,要看cli.py写了什么

加载模型,等待几分钟

python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [03:17<00:00, 98.73s/it]

推理:

USER: Introduce China

ASSISTANT: China, officially known as the People's Republic of China (PRC), is a sovereign state located in East Asia. It is the world's most populous country, with a population of over 1.4 billion people, and the third-largest country by land area, covering approximately 9.6 million square kilometers.

China has a long and complex history, dating back more than 5,000 years. It has been ruled by various dynasties, including the Zhou, Qin, Han, Tang, Song, Yuan, Ming, and Qing dynasties, among others.

The People's Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the People's Republic of China, which was adopted in 1982. The government is divided into various branches, including the National People's Congress (NPC), the State Council, the Supreme People's Court, and the Central Military Commission.

China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.

The People's Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the People's Republic of China, which was adopted in 1982. The government is divided into various branches, including the National People's Congress (NPC), the State Council, the Supreme People's Court, and the Central Military Commission.

China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.

The economy of China has experienced rapid growth in recent decades, making it the world's second-largest economy by nominal GDP and the largest by purchasing power parity (PPP). The country is a leading global exporter of goods and services, with industries such as electronics, textiles, automobiles, and telecommunications playing a significant role in its economic development.

China has been actively involved in international affairs, participating as a permanent member of the United Nations Security Council and a member of various multilateral organizations, including the World Trade Organization (WTO), the Asia-Pacific Economic Cooperation (APEC), and the Group of 20 (G20) major economies.

USER: china

ASSISTANT: China is a country located in East Asia, covering approximately 9.6 million square kilometers. With a population of over 1.4 billion people, it is the most populous country in the world. The official language is Mandarin Chinese, and the capital city is Beijing. China has a rich history and a diverse cultural heritage, with many different ethnic groups and over 56 recognized ethnic minorities.

目前尚不能支持中文提问和中文回答

提问方式很重要:输入china,回答的只能说是相关,文字就少很多。输入 Introduce China,输出大段的介绍

相关推荐
Guofu_Liao4 小时前
大语言模型---LoRA简介;LoRA的优势;LoRA训练步骤;总结
人工智能·语言模型·自然语言处理·矩阵·llama
AI_小站13 小时前
RAG 示例:使用 langchain、Redis、llama.cpp 构建一个 kubernetes 知识库问答
人工智能·程序人生·langchain·kubernetes·llama·知识库·rag
Guofu_Liao13 小时前
Llama模型文件介绍
人工智能·llama
Donvink17 小时前
多模态大语言模型——《动手学大模型》实践教程第六章
人工智能·深度学习·语言模型·自然语言处理·llama
Donvink20 小时前
大模型安全和越狱攻击——《动手学大模型》实践教程第五章
深度学习·安全·语言模型·llama
Donvink21 小时前
大模型智能体安全——《动手学大模型》实践教程第七章
深度学习·安全·语言模型·prompt·llama
慢热型网友.1 天前
【项目实战】基于 LLaMA-Factory 通过 LoRA 微调 Qwen2
llama
机器学习是魔鬼1 天前
LLaMA-Factory 上手即用教程
llama·模型训练·ai功能岛·矩池云
Galeoto1 天前
fine tuning with llama-factory
llama
Struart_R2 天前
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models 论文解读
语言模型·llama·多模态·三维生成·自回归