llama-index调用qwen大模型实现RAG

背景

llama-index在实现RAG方案的时候多是用的llama等英文大模型，对于国内的诸多模型案例较少，本次将使用qwen大模型实现llama-index的RAG方案。

环境配置

（1）pip包

llamaindex需要预装很多包，这里先把我成功的案例里面的pip包配置发出来，在requirements.txt里面。

absl-py==1.4.0
accelerate==0.27.2
aiohttp==3.9.3
aiosignal==1.3.1
aliyun-python-sdk-core==2.13.36
aliyun-python-sdk-kms==2.16.1
annotated-types==0.6.0
anyio==3.7.1
apphub @ file:///environment/apps/apphub/dist/apphub-1.0.0.tar.gz#sha256=260f99c0de4c575b19ab913aa134877e9efd81b820b97511fc8379674643c253
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asgiref==3.7.2
asttokens==2.2.1
astunparse==1.6.3
async-timeout==4.0.3
attrs==23.1.0
Babel==2.12.1
backcall==0.2.0
backoff==2.2.1
bcrypt==4.1.2
beautifulsoup4==4.12.3
bleach==6.0.0
boltons @ file:///croot/boltons_1677628692245/work
brotlipy==0.7.0
bs4==0.0.2
build==1.1.1
cachetools==5.3.1
certifi @ file:///croot/certifi_1690232220950/work/certifi
cffi @ file:///croot/cffi_1670423208954/work
chardet==3.0.4
charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
chroma-hnswlib==0.7.3
chromadb==0.4.24
click==7.1.2
cmake==3.25.0
coloredlogs==15.0.1
comm==0.1.4
conda @ file:///croot/conda_1690494963117/work
conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work
conda-libmamba-solver @ file:///croot/conda-libmamba-solver_1685032319139/work/src
conda-package-handling @ file:///croot/conda-package-handling_1685024767917/work
conda_package_streaming @ file:///croot/conda-package-streaming_1685019673878/work
contourpy==1.2.0
crcmod==1.7
cryptography @ file:///croot/cryptography_1686613057838/work
cycler==0.12.1
dataclasses-json==0.6.4
debugpy==1.6.7
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
dirtyjson==1.0.8
distro==1.9.0
ecdsa==0.18.0
exceptiongroup==1.1.2
executing==1.2.0
fastapi==0.104.1
fastjsonschema==2.18.0
featurize==0.0.24
filelock==3.9.0
flatbuffers==23.5.26
fonttools==4.44.0
frozenlist==1.4.1
fsspec==2024.2.0
gast==0.4.0
google-auth==2.22.0
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
googleapis-common-protos==1.62.0
greenlet==3.0.3
grpcio==1.62.0
gunicorn==21.2.0
h11==0.14.0
h5py==3.9.0
httpcore==0.17.3
httptools==0.6.1
httpx==0.24.1
huggingface-hub==0.20.3
humanfriendly==10.0
idna==2.10
imageio==2.32.0
importlib-metadata==6.11.0
importlib_resources==6.1.3
ipykernel==6.25.0
ipython==8.14.0
ipython-genutils==0.2.0
ipywidgets==8.1.2
jedi==0.19.0
Jinja2==3.1.2
jmespath==0.10.0
joblib==1.3.2
json5==0.9.14
jsonpatch @ file:///tmp/build/80754af9/jsonpatch_1615747632069/work
jsonpointer==2.1
jsonschema==4.18.6
jsonschema-specifications==2023.7.1
jupyter-server==1.24.0
jupyter_client==8.3.0
jupyter_core==5.3.1
jupyterlab==3.2.9
jupyterlab-pygments==0.2.2
jupyterlab_server==2.24.0
jupyterlab_widgets==3.0.10
keras==2.13.1
kiwisolver==1.4.5
kubernetes==29.0.0
lazy_loader==0.3
libclang==16.0.6
libmambapy @ file:///croot/mamba-split_1685993156657/work/libmambapy
lit==15.0.7
llama-index==0.10.17
llama-index-agent-openai==0.1.5
llama-index-cli==0.1.8
llama-index-core==0.10.17
llama-index-embeddings-huggingface==0.1.4
llama-index-embeddings-openai==0.1.6
llama-index-indices-managed-llama-cloud==0.1.3
llama-index-legacy==0.9.48
llama-index-llms-huggingface==0.1.3
llama-index-llms-openai==0.1.7
llama-index-multi-modal-llms-openai==0.1.4
llama-index-program-openai==0.1.4
llama-index-question-gen-openai==0.1.3
llama-index-readers-file==0.1.8
llama-index-readers-llama-parse==0.1.3
llama-index-vector-stores-chroma==0.1.5
llama-parse==0.3.8
llamaindex-py-client==0.1.13
Markdown==3.4.4
MarkupSafe==2.1.2
marshmallow==3.21.1
matplotlib==3.8.1
matplotlib-inline==0.1.6
mistune==3.0.1
mmh3==4.1.0
monotonic==1.6
mpmath==1.2.1
multidict==6.0.4
mypy-extensions==1.0.0
nbclassic==0.2.8
nbclient==0.8.0
nbconvert==7.7.3
nbformat==5.9.2
nest-asyncio==1.6.0
networkx==3.0
nltk==3.8.1
notebook==6.4.12
numpy==1.24.1
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.4.99
nvidia-nvtx-cu12==12.1.105
oauthlib==3.2.2
onnxruntime==1.17.1
openai==1.13.3
opencv-python==4.8.1.78
opentelemetry-api==1.23.0
opentelemetry-exporter-otlp-proto-common==1.23.0
opentelemetry-exporter-otlp-proto-grpc==1.23.0
opentelemetry-instrumentation==0.44b0
opentelemetry-instrumentation-asgi==0.44b0
opentelemetry-instrumentation-fastapi==0.44b0
opentelemetry-proto==1.23.0
opentelemetry-sdk==1.23.0
opentelemetry-semantic-conventions==0.44b0
opentelemetry-util-http==0.44b0
opt-einsum==3.3.0
orjson==3.9.15
oss2==2.18.1
overrides==7.7.0
packaging @ file:///croot/packaging_1678965309396/work
pandas==2.1.2
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
platformdirs==3.10.0
pluggy @ file:///tmp/build/80754af9/pluggy_1648024709248/work
posthog==3.5.0
prometheus-client==0.17.1
prompt-toolkit==3.0.39
protobuf==4.23.4
psutil==5.9.5
ptyprocess==0.7.0
pulsar-client==3.4.0
pure-eval==0.2.2
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycosat @ file:///croot/pycosat_1666805502580/work
pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
pycryptodome==3.18.0
pydantic==2.4.2
pydantic_core==2.10.1
Pygments==2.15.1
PyMuPDF==1.23.26
PyMuPDFb==1.23.22
pyOpenSSL @ file:///croot/pyopenssl_1677607685877/work
pyparsing==3.1.1
pypdf==4.1.0
PyPika==0.48.9
pyproject_hooks==1.0.0
PySocks @ file:///home/builder/ci_310/pysocks_1640793678128/work
python-dateutil==2.8.2
python-dotenv==1.0.0
pytz==2023.3.post1
PyYAML==6.0.1
pyzmq==25.1.0
referencing==0.30.0
regex==2023.12.25
requests==2.31.0
requests-oauthlib==1.3.1
rpds-py==0.9.2
rsa==4.9
ruamel.yaml @ file:///croot/ruamel.yaml_1666304550667/work
ruamel.yaml.clib @ file:///croot/ruamel.yaml.clib_1666302247304/work
safetensors==0.4.2
scikit-image==0.22.0
scikit-learn==1.3.2
scipy==1.11.3
seaborn==0.13.0
Send2Trash==1.8.2
six @ file:///tmp/build/80754af9/six_1644875935023/work
sniffio==1.3.0
socksio==1.0.0
soupsieve==2.4.1
SQLAlchemy==2.0.28
sshpubkeys==3.3.1
stack-data==0.6.2
starlette==0.27.0
sympy==1.11.1
tabulate==0.8.7
tenacity==8.2.3
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorflow==2.13.0
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.33.0
termcolor==2.3.0
terminado==0.17.1
threadpoolctl==3.2.0
tifffile==2023.9.26
tiktoken==0.6.0
tinycss2==1.2.1
tokenizers==0.15.2
tomli==2.0.1
toolz @ file:///croot/toolz_1667464077321/work
torch==2.2.1
torchaudio==2.0.2+cu118
torchvision==0.15.2+cu118
tornado==6.3.2
tqdm==4.66.2
traitlets==5.9.0
transformers==4.38.2
triton==2.2.0
typer==0.9.0
typing-inspect==0.9.0
typing_extensions==4.8.0
tzdata==2023.3
urllib3==1.25.11
uvicorn==0.23.2
uvloop==0.19.0
watchfiles==0.21.0
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.2.1
websockets==12.0
Werkzeug==2.3.6
widgetsnbextension==4.0.10
workspace @ file:///home/featurize/work/workspace/dist/workspace-0.1.0.tar.gz#sha256=b292beb3599f79d3791771eff9dc422cc37c58c1fc8daadeafbf025a2e7ea986
wrapt==1.15.0
yarl==1.9.2
zipp==3.17.0
zstandard @ file:///croot/zstandard_1677013143055/work

（2）python 环境

（3）安装命令

!pip install llama-index
!pip install llama-index-llms-huggingface
!pip install llama-index-embeddings-huggingface
!pip install llama-index ipywidgets
!pip install torch
!git clone https://www.modelscope.cn/AI-ModelScope/bge-small-zh-v1.5.git
!git clone https://www.modelscope.cn/qwen/Qwen1.5-4B-Chat.git

（4）目录结构

代码

（1）加载模型

import torch
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core import PromptTemplate
import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'
# Model names (make sure you have access on HF)
LLAMA2_7B = "/home/featurize/Qwen1.5-4B-Chat"
# LLAMA2_7B_CHAT = "meta-llama/Llama-2-7b-chat-hf"
# LLAMA2_13B = "meta-llama/Llama-2-13b-hf"
LLAMA2_13B_CHAT = "/home/featurize/Qwen1.5-4B-Chat"
# LLAMA2_70B = "meta-llama/Llama-2-70b-hf"
# LLAMA2_70B_CHAT = "meta-llama/Llama-2-70b-chat-hf"

selected_model = LLAMA2_13B_CHAT

SYSTEM_PROMPT = """You are an AI assistant that answers questions in a friendly manner, based on the given source documents. Here are some rules you always follow:
- Generate human readable output, avoid creating output with gibberish text.
- Generate only the requested output, don't include any other language before or after the requested output.
- Never say thank you, that you are happy to help, that you are an AI agent, etc. Just answer directly.
- Generate professional language typically used in business documents in North America.
- Never generate offensive or foul language.
"""

query_wrapper_prompt = PromptTemplate(
    "[INST]<<SYS>>\n" + SYSTEM_PROMPT + "<</SYS>>\n\n{query_str}[/INST] "
)

llm = HuggingFaceLLM(context_window=4096,
    max_new_tokens=2048,
    generate_kwargs={"temperature": 0.0, "do_sample": False},
    query_wrapper_prompt=query_wrapper_prompt,
    tokenizer_name=selected_model,
    model_name=selected_model,
    device_map="auto"
)

（2）加载词嵌入向量

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

embed_model = HuggingFaceEmbedding(model_name="/home/featurize/bge-small-zh-v1.5")

from llama_index.core import Settings

Settings.llm = llm
Settings.embed_model = embed_model

from llama_index.core import SimpleDirectoryReader

# load documents
documents = SimpleDirectoryReader("./data/").load_data()

from llama_index.core import VectorStoreIndex
index = VectorStoreIndex.from_documents(documents)

index

# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine()

response = query_engine.query("小额贷款咋规定的?")
print(response)

知识库

llamaindex实现RAG中很关键的一环就是知识库，知识库主要是各种类型的文档，这里给的文档是一个pdf文件，文件内容如下。

总结

从上面的代码可以看出，我们使用qwen和bge-zh模型可以实现本地下载模型的RAG方案，知识库里面的内容也可以实现中文问答，这非常有利于我们进行私有化部署方案，从而扩展我们的功能。