Question mutiple pdf‘s using openai, pinecone, langchain

题意:使用 OpenAI、Pinecone 和 LangChain 对多个 PDF 文件进行提问。

问题背景:

I am trying to ask questions against a multiple pdf using pinecone and openAI but I dont know how to.

我正在尝试使用 Pinecone 和 OpenAI 对多个 PDF 文件进行提问,但我不知道该怎么做。

The code below works for asking questions against one document. but I would like to have multiple documents to ask questions against:

下面的代码可以用于对一个文档进行提问,但我想要能够对多个文档提问:

python 复制代码
# process_message.py
from flask import request
import pinecone
# from PyPDF2 import PdfReader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import ElasticVectorSearch, Pinecone, Weaviate, FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
import os
import json
# from constants.company import file_company_id_column, file_location_column, file_name_column
from services.files import FileFireStorage
from middleware.auth import check_authorization
import configparser
from langchain.document_loaders import UnstructuredPDFLoader, OnlinePDFLoader, PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter


def process_message():
    
    # Create a ConfigParser object and read the config.ini file
    config = configparser.ConfigParser()
    config.read('config.ini')
    # Retrieve the value of OPENAI_API_KEY
    openai_key = config.get('openai', 'OPENAI_API_KEY')
    pinecone_env_key = config.get('pinecone', 'PINECONE_ENVIRONMENT')
    pinecone_api_key = config.get('pinecone', 'PINECONE_API_KEY')


    loader = PyPDFLoader("docs/ops.pdf")
    data = loader.load()
    # data = body['data'][1]['name']
    # Print information about the loaded data
    print(f"You have {len(data)} document(s) in your data")
    print(f"There are {len(data[30].page_content)} characters in your document")

    # Chunk your data up into smaller documents
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=0)
    texts = text_splitter.split_documents(data)
   

    embeddings = OpenAIEmbeddings(openai_api_key=openai_key)

    pinecone.init(api_key=pinecone_api_key, environment=pinecone_env_key)
    index_name = "pdf-chatbot"  # Put in the name of your Pinecone index here

    docsearch = Pinecone.from_texts([t.page_content for t in texts], embeddings, index_name=index_name)
    # Query those docs to get your answer back
    llm = OpenAI(temperature=0, openai_api_key=openai_key)
    chain = load_qa_chain(llm, chain_type="stuff")

    query = "Are there any other documents listed in this document?"
    docs = docsearch.similarity_search(query)
    answer = chain.run(input_documents=docs, question=query)
    print(answer)

    return answer

I added as many comments as I could there. I got this information from

我在代码中添加了尽可能多的注释。我从以下来源获取了这些信息:https://www.youtube.com/watch?v=h0DHDp1FbmQ

I tried to look at other stackoverflow questions about this but could not find anything similar

我试图查看其他与此相关的 Stack Overflow 问题,但没有找到类似的内容。

问题解决:

You can load multiple PDFS with PyPDFDirectoryLoader

你可以使用 `PyPDFDirectoryLoader` 加载多个 PDF 文件。

相关推荐
gb421528719 分钟前
python中pypdf库和langchain-unstructured库在解析pdf文件的时候的区别?
python·langchain·pdf
weixin1997010801625 分钟前
☁️《抖店API基础¥0.018/百次·增值¥0.05/百次:云内云外价差架构实战》(附Python源码)
开发语言·python·架构
0566461 小时前
Python数据结构——循环队列
python·学习
萌动的小火苗1 小时前
1、python基础面试题
java·开发语言·python
ElasticPDF-新国产PDF编辑器2 小时前
【最新·免费PDF编辑器·不限终端·最高性价比SDK】高亮、评论、图章、手绘与签名,一套完成 PDF 批注
pdf·编辑器
sunywz2 小时前
【从零搭建物联网智能充电桩系统】2、自定义二进制协议:设备为什么不用 JSON?
python·物联网·json
小叮当爱咖啡2 小时前
Day3.参数+Prompt三板斧
开发语言·python·prompt
badhope2 小时前
用RAG做了个智能客服,上线第一天就被用户骂了——我的7天实战复盘
人工智能·langchain
菜冻鱼2 小时前
Python-pandas-索引与筛选
开发语言·笔记·python·numpy·pandas·学习方法
青 春 记 忆2 小时前
LeetCode 53. 最大子数组和|Python 解法详解
python·算法·leetcode