RAG 进阶 LlamaIndex多文档

我在LlamaIndex和LangChain框架学习中,都有玩过RAG。最近准备开个RAG进阶系统,一起学习AI, 一起成长。欢迎点赞,留言交流。

前言

我们一起来学习LlamaIndex功能更完善的多文档RAG,大家可以参照官方文档来对比学习。

Advanced RAG - LlamaIndex Multi-Doc Agent

  • 单个文档RAG Agent流程

我们结合上图来理解一下文档的处理流程。Document就是文档,它会被解析成多个数据节点(Node),类似RAG 进阶 半结构化数据中element的概念。LlamaIndex确实在文档RAG这块有它的优势,在这块LangChain可以将阵地交给它。

当文档被分割为节点后,我们开始创建索引。索引包括Summary Index和Vector Index。摘要索引是每个节点的摘要,每个节点都有摘要索引,方便基于摘要进行检索。Vector Index是对文档的索引和向量存储。

有了索引,我们就可以基于LlamaIndex提供的工具封装成Query Engine Tool,形式是函数。

最后GPT-4根据情况调用相应的Query Engine Tool 函数, 完成工作。我们就为用户提供了文档RAG检索的Agent(代理)。

  • 多个文档RAG Agent 流程

现在我们有多份文档,每份文档根据上面的流程会被包装成独立的Agent。

在上图中,每个Query Engine Tool由一个叫Object Index的对象索引来管理。Object Index 了解每个Query Engine Tool的内容,到时它知道应该调用哪个文档。有了Object Index 我们就可以实例化Retriever, 生成Agent For QA,结合LLM一起work。

我们看下QA Agent的prmpt。他告诉LLM, 总是使用Object Index管理的Query Engine Tools回答问题。

实战

在这个例子中,我们会使用到一下LlamaIndex的组件

  • VectorStoreIndex 文本数据的向量化做的存储
  • SummaryIndex 文本摘要信息做的索引
  • ObjectIndex 对QueryEngineTool和Agent进行的索引
  • QueryEngineTool 查询引擎工具的实现类
  • OpenAIAgent openai 代理
  • FnRetrieverOpenAIAgent 检索器代理

Let's coding!

  • 安装LlamaIndex
css 复制代码
!pip install llama-index -q -U
  • 准备openai
lua 复制代码
import os 
import openai

os.environ["OPENAI_API_KEY"]="your valid openai api key"
openai.api_key = os.environ["OPENAI_API_KEY]
  • LlamaIndex 组件引入
javascript 复制代码
from llama_index import(
    VectorStoreIndex,
    SummaryIndex,
    SimpleDirectoryReader,
    ServiceContext
)
from llama_index.tools import QueryEngineTool, ToolMetaData
from llama_index.llms import OpenAI
  • 文档

我们使用的是欧洲五大联赛维基百科的内容,多文档。RAG要实现的功能能回答五大联赛相关的问题,而且不仅仅是单个联赛,还可以针对多个联赛回答。比如: 请比较英超联赛和西甲联赛的历史,以及在冠军杯的表现。非常期待跨多个文档检索,再合成...

ini 复制代码
wiki_titles = [
    "Serie A",
    "Premier League",
    "Bundesliga",
    "La Liga",
    "Ligue 1"
]
python 复制代码
#下载文档
from pathlib import Path
import requests

for title in wiki_titles:
    response = requests.get(
        "https://en.wikipedia.org/w/api.php",
        params={
            "action":"query",
            "format":"json",
            "titles": title,
            "prop":"extract",
            "explaintext":True,
        },
    ).json()
    page = next(iter(response["query"]["pages"].values()))
    wiki_text = page['extract']
    
    data_path=Path["data"]
    if not data_path.exists():
        Path.mkdir(data_path)
    
    with open(data_path / f"{title}.txt", "w") as fp:
        fp.write(wiki_text)
  • 加载文档
ini 复制代码
    leagues_docs = {}
    for wiki_title in wiki_titles:
        leagues_docs[wiki_title] = SimpleDirectoryReader(
            input_files=[f"data/{wiki_title}.txt"]
        ).load_data()
    # gpt-3.5-turbo对文本处理已经足够了
    llm = OpenAI(temperature=0, model="gpt-3.5-turbo")
    service_context = ServiceContext.from_defaults(llm=llm)
  • 构建summarize索引和Vector索引
ini 复制代码
# OpenAI代理
from llama_index.agent import OpenAIAgent 
from llama_index import load_index_from_storage, StorageContext 
# 分割器
from llama_index.node_parser import SentenceSplitter 
import os 
# 实例化分割器
node_parser = SentenceSplitter() # Build agents dictionary 
# agents dict
agents = {} 
# 查询引擎 dict
query_engines = {} # this is for the baseline 
# 所有的节点
all_nodes = [] 
for idx, wiki_title in enumerate(wiki_titles): 
    nodes = node_parser.get_nodes_from_documents(leagues_docs[wiki_title]) 
    all_nodes.extend(nodes) 
    if not os.path.exists(f"./data/{wiki_title}"):
        # build vector index 
        vector_index = VectorStoreIndex(nodes, 
        service_context=service_context) 
        vector_index.storage_context.persist(
            persist_dir=f"./data/{wiki_title}"
        )
    else:
        vector_index = load_index_from_storage( StorageContext.from_defaults(persist_dir=f"./data/{wiki_title}"), service_context=service_context, )
   summary_index = SummaryIndex(nodes, service_context=service_context)
   vector_query_engine = vector_index.as_query_engine()
   summary_query_engine = summary_index.as_query_engine()
   
   query_engine_tools = [ QueryEngineTool( query_engine=vector_query_engine, metadata=ToolMetadata( name="vector_tool", description=( "Useful for questions related to specific aspects of" f" {wiki_title} (e.g. the history, teams " "and performance in EU, or more)." ), ), ), '
   QueryEngineTool( query_engine=summary_query_engine, metadata=ToolMetadata( name="summary_tool", description=( "Useful for any requests that require a holistic summary" f" of EVERYTHING about {wiki_title}. For questions about" " more specific sections, please use the vector_tool." ), ), ), ]
   
   # build agent 
   function_llm = OpenAI(model="gpt-4") 
   agent = OpenAIAgent.from_tools( 
   query_engine_tools, 
   llm=function_llm, 
   verbose=True, 
   system_prompt=f"""\ You are a specialized agent designed to answer queries about {wiki_title}. You must ALWAYS use at least one of the tools provided when answering a question; do NOT rely on prior knowledge.\ """, ) 
   agents[wiki_title] = agent 
   query_engines[wiki_title] = vector_index.as_query_engine( similarity_top_k=2 )
  • tools
ini 复制代码
all_tools = [] for wiki_title in wiki_titles: 
    wiki_summary = ( f"This content contains Wikipedia articles about {wiki_title}. Use" f" this tool if you want to answer any questions about {wiki_title}.\n" ) 
    doc_tool = QueryEngineTool( query_engine=agents[wiki_title], 
    metadata=ToolMetadata( name=f"tool_{wiki_title.replace(' ', '_')}", 
    description=wiki_summary, ), ) 
    all_tools.append(doc_tool)
  • objct index
ini 复制代码
from llama_index import VectorStoreIndex 
from llama_index.objects import ObjectIndex, SimpleToolNodeMapping 
tool_mapping = SimpleToolNodeMapping.from_objects(all_tools) 
obj_index = ObjectIndex.from_objects( all_tools, tool_mapping, VectorStoreIndex, )
  • retriever agent
ini 复制代码
from llama_index.agent import FnRetrieverOpenAIAgent 
top_agent = FnRetrieverOpenAIAgent.from_retriever( 
obj_index.as_retriever(similarity_top_k=3), 
system_prompt=""" \ You are an agent designed to answer queries about the European top football leagues. Please always use the tools provided to answer a question. Do not rely on prior knowledge.\ """, verbose=True, )
  • 查询
ini 复制代码
from llama_index.agent import FnRetrieverOpenAIAgent 
top_agent = FnRetrieverOpenAIAgent.from_retriever( obj_index.as_retriever(similarity_top_k=3), system_prompt=""" \ You are an agent designed to answer queries about the European top football leagues. Please always use the tools provided to answer a question. Do not rely on prior knowledge.\ """, verbose=True, )
  • 打印结果
vbnet 复制代码
La Liga has a rich history dating back to the 1930s. During the early years, Athletic Club was the dominant team, but in the 1940s, Atlético Madrid, Barcelona, and Valencia emerged as strong contenders. The 1950s saw the rise of FC Barcelona and the dominance of Real Madrid, which continued into the 1960s and 1970s.

In terms of UEFA Champions League (UCL) performance, La Liga teams have had a significant impact. Real Madrid, Barcelona, and Atlético Madrid have been particularly successful, with multiple UCL titles to their names. Other Spanish clubs like Sevilla and Valencia have also won international trophies. La Liga consistently sends a significant number of teams to the Champions League group stage, showcasing the league's overall strength and competitiveness.

总结

  • LlamaIndex 为多文档RAG提供了VectorStoreIndex、SummaryIndex、ObjectIndex、QueryEngineTool、FnRetrieverOpenAIAgent 等丰富组件,过程复杂,但井然有序
  • 除VectorStoreIndex、SummaryIndex,还有agent 索引, object 索引
  • 相应的prompt 设计,让用户的问题,由相应的 tool来查询 相应的索引,最后再合并起来,cool

参考资料

相关推荐
HPC_fac130520678161 小时前
科研深度学习:如何精选GPU以优化服务器性能
服务器·人工智能·深度学习·神经网络·机器学习·数据挖掘·gpu算力
猎嘤一号2 小时前
个人笔记本安装CUDA并配合Pytorch使用NVIDIA GPU训练神经网络的计算以及CPUvsGPU计算时间的测试代码
人工智能·pytorch·神经网络
天润融通2 小时前
天润融通携手挚达科技:AI技术重塑客户服务体验
人工智能
Elastic 中国社区官方博客4 小时前
使用 Elastic AI Assistant for Search 和 Azure OpenAI 实现从 0 到 60 的转变
大数据·人工智能·elasticsearch·microsoft·搜索引擎·ai·azure
江_小_白5 小时前
自动驾驶之激光雷达
人工智能·机器学习·自动驾驶
yusaisai大鱼6 小时前
TensorFlow如何调用GPU?
人工智能·tensorflow
珠海新立电子科技有限公司9 小时前
FPC柔性线路板与智能生活的融合
人工智能·生活·制造
IT古董9 小时前
【机器学习】机器学习中用到的高等数学知识-8. 图论 (Graph Theory)
人工智能·机器学习·图论
曼城周杰伦9 小时前
自然语言处理:第六十三章 阿里Qwen2 & 2.5系列
人工智能·阿里云·语言模型·自然语言处理·chatgpt·nlp·gpt-3
余炜yw10 小时前
【LSTM实战】跨越千年,赋诗成文:用LSTM重现唐诗的韵律与情感
人工智能·rnn·深度学习