LLM之基于llama-index部署本地embedding与GLM-4模型并初步搭建RAG（其他大模型也可，附上ollma方式运行）

黑夜寻白天2024-08-23 22:40

前言

日常没空，留着以后写

llama-index简介

官网：https://docs.llamaindex.ai/en/stable/

简介也没空，以后再写

注：先说明，随着官方的变动，代码也可能变动，大家运行不起来，可以进官网查查资料

加载本地embedding模型

如果没有找到 llama_index.embeddings.huggingface

那么：pip install llama_index-embeddings-huggingface

还不行进入官网，输入huggingface进行搜索

python 复制代码 
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings

Settings.embed_model = HuggingFaceEmbedding(
    model_name=f"{embed_model_path}",device='cuda'

)

加载本地LLM模型

还是那句话，如果以下代码不行，进官网搜索Custom LLM Model

python 复制代码 
from llama_index.core.llms import (
    CustomLLM,
    CompletionResponse,
    CompletionResponseGen,
    LLMMetadata,
)
from llama_index.core.llms.callbacks import llm_completion_callback
from transformers import AutoTokenizer, AutoModelForCausalLM

class GLMCustomLLM(CustomLLM):
    context_window: int = 8192  # 上下文窗口大小
    num_output: int = 8000  # 输出的token数量
    model_name: str = "glm-4-9b-chat"  # 模型名称
    tokenizer: object = None  # 分词器
    model: object = None  # 模型
    dummy_response: str = "My response"

    def __init__(self, pretrained_model_name_or_path):
        super().__init__()

        # GPU方式加载模型
        self.tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path, device_map="cuda", trust_remote_code=True)
        self.model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, device_map="cuda", trust_remote_code=True).eval()

        # CPU方式加载模型
        # self.tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path, device_map="cpu", trust_remote_code=True)
        # self.model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, device_map="cpu", trust_remote_code=True)
        self.model = self.model.float()

    @property
    def metadata(self) -> LLMMetadata:
        """Get LLM metadata."""
        # 得到LLM的元数据
        return LLMMetadata(
            context_window=self.context_window,
            num_output=self.num_output,
            model_name=self.model_name,
        )

    # @llm_completion_callback()
    # def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
    #     return CompletionResponse(text=self.dummy_response)
    #
    # @llm_completion_callback()
    # def stream_complete(
    #     self, prompt: str, **kwargs: Any
    # ) -> CompletionResponseGen:
    #     response = ""
    #     for token in self.dummy_response:
    #         response += token
    #         yield CompletionResponse(text=response, delta=token)

    @llm_completion_callback()  # 回调函数
    def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
        # 完成函数
        print("完成函数")

        inputs = self.tokenizer.encode(prompt, return_tensors='pt').cuda()  # GPU方式
        # inputs = self.tokenizer.encode(prompt, return_tensors='pt')  # CPU方式
        outputs = self.model.generate(inputs, max_length=self.num_output)
        response = self.tokenizer.decode(outputs[0])
        return CompletionResponse(text=response)

    @llm_completion_callback()
    def stream_complete(
        self, prompt: str, **kwargs: Any
    ) -> CompletionResponseGen:
        # 流式完成函数
        print("流式完成函数")

        inputs = self.tokenizer.encode(prompt, return_tensors='pt').cuda()  # GPU方式
        # inputs = self.tokenizer.encode(prompt, return_tensors='pt')  # CPU方式
        outputs = self.model.generate(inputs, max_length=self.num_output)
        response = self.tokenizer.decode(outputs[0])
        for token in response:
            yield CompletionResponse(text=token, delta=token)

基于本地模型搭建简易RAG

python 复制代码 
from typing import Any

from llama_index.core.llms import (
    CustomLLM,
    CompletionResponse,
    CompletionResponseGen,
    LLMMetadata,
)
from llama_index.core.llms.callbacks import llm_completion_callback
from transformers import AutoTokenizer, AutoModelForCausalLM
from llama_index.core import Settings,VectorStoreIndex,SimpleDirectoryReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding


class GLMCustomLLM(CustomLLM):
    context_window: int = 8192  # 上下文窗口大小
    num_output: int = 8000  # 输出的token数量
    model_name: str = "glm-4-9b-chat"  # 模型名称
    tokenizer: object = None  # 分词器
    model: object = None  # 模型
    dummy_response: str = "My response"

    def __init__(self, pretrained_model_name_or_path):
        super().__init__()

        # GPU方式加载模型
        self.tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path, device_map="cuda", trust_remote_code=True)
        self.model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, device_map="cuda", trust_remote_code=True).eval()

        # CPU方式加载模型
        # self.tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path, device_map="cpu", trust_remote_code=True)
        # self.model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, device_map="cpu", trust_remote_code=True)
        self.model = self.model.float()

    @property
    def metadata(self) -> LLMMetadata:
        """Get LLM metadata."""
        # 得到LLM的元数据
        return LLMMetadata(
            context_window=self.context_window,
            num_output=self.num_output,
            model_name=self.model_name,
        )

    # @llm_completion_callback()
    # def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
    #     return CompletionResponse(text=self.dummy_response)
    #
    # @llm_completion_callback()
    # def stream_complete(
    #     self, prompt: str, **kwargs: Any
    # ) -> CompletionResponseGen:
    #     response = ""
    #     for token in self.dummy_response:
    #         response += token
    #         yield CompletionResponse(text=response, delta=token)

    @llm_completion_callback()  # 回调函数
    def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
        # 完成函数
        print("完成函数")

        inputs = self.tokenizer.encode(prompt, return_tensors='pt').cuda()  # GPU方式
        # inputs = self.tokenizer.encode(prompt, return_tensors='pt')  # CPU方式
        outputs = self.model.generate(inputs, max_length=self.num_output)
        response = self.tokenizer.decode(outputs[0])
        return CompletionResponse(text=response)

    @llm_completion_callback()
    def stream_complete(
        self, prompt: str, **kwargs: Any
    ) -> CompletionResponseGen:
        # 流式完成函数
        print("流式完成函数")

        inputs = self.tokenizer.encode(prompt, return_tensors='pt').cuda()  # GPU方式
        # inputs = self.tokenizer.encode(prompt, return_tensors='pt')  # CPU方式
        outputs = self.model.generate(inputs, max_length=self.num_output)
        response = self.tokenizer.decode(outputs[0])
        for token in response:
            yield CompletionResponse(text=token, delta=token)


if __name__ == "__main__":


    # 定义你的LLM
    pretrained_model_name_or_path = r'/home/nlp/model/LLM/THUDM/glm-4-9b-chat'
    embed_model_path = '/home/nlp/model/Embedding/BAAI/bge-m3'

    Settings.embed_model = HuggingFaceEmbedding(
        model_name=f"{embed_model_path}",device='cuda'

    )

    Settings.llm = GLMCustomLLM(pretrained_model_name_or_path)

    documents = SimpleDirectoryReader(input_dir="home/xxxx/input").load_data()
    index = VectorStoreIndex.from_documents(
        documents,
    )


    # 查询和打印结果
    query_engine = index.as_query_engine()
    response = query_engine.query("萧炎的表妹是谁?")

    print(response)

ollama

python 复制代码 
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama

documents = SimpleDirectoryReader("data").load_data()

# bge-base embedding model
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")

# ollama
Settings.llm = Ollama(model="llama3", request_timeout=360.0)

index = VectorStoreIndex.from_documents(
    documents,
)

欢迎大家点赞或收藏

大家的点赞或收藏可以鼓励作者加快更新哟~

参加链接：

LlamaIndex中的CustomLLM（本地加载模型）
llamaIndex 基于GPU加载本地embedding模型

官网文档

官网_starter_example_loca

官网_usage_custom

相关推荐
汪汪汪侠客3 天前
源码解析（一）：GraphRAG
算法·面试·大模型·rag·graphrag
哥不是小萝莉3 天前
如何实现RAG与MCP集成
ai·rag
勇敢牛牛_5 天前
近期知识库开发过程中遇到的一些问题
rust·知识库·rag
明明跟你说过5 天前
深入 RAG（检索增强生成）系统架构：如何构建一个能查资料的大语言模型系统
人工智能·语言模型·自然语言处理·embedding·rag
wangwren6 天前
SpringAI--RAG知识库
java·rag·spring ai·pgvector
正在走向自律6 天前
AI新手入门解锁元生代MaaS平台：API工作流调用全攻略
人工智能·自然语言处理·知识库·rag·maas·openai 兼容接口
AI小白龙*6 天前
重磅发布 | 复旦533页《大规模语言模型：从理论到实践（第2版）》（免费下载）
人工智能·程序员·llm·ai大模型·rag
在未来等你7 天前
互联网大厂Java求职面试：AI大模型推理服务性能优化与向量数据库分布式检索
java·llm·milvus·向量数据库·rag·spring ai·语义缓存
WinddyAkoky7 天前
【RAG】RAG综述|一文了解RAG|从零开始（上）
rag
fengchengwu20127 天前
python调用langchain实现RAG
python·langchain·rag·dashscope·pgvector
热门推荐
01机器学习实验--- 金融数据基础与计算在线实验闯关02KGG转MP3工具|非KGM文件|解密音频03YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】04从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑05【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！06DeepSeek各版本说明与优缺点分析07【解决】Android Gradle Sync 报错 Could not read workspace metadata08【2025年最新】OpenWrt 更换国内源的指南（图形界面版)09组基轨迹建模 GBTM的介绍与实现（Stata 或 R）10山东大学2024深度学习期末考试回忆