Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain。
开源网址：https://github.com/chatchat-space/Langchain-Chatchat
最近因工作需要，想体验一下ChatGLM的功能，就用手头的think P70笔记本安装部署了一下，自己记录一下，因为网上大部分的作者可能是NLP大咖，所以会忽略一些基础的东西，自己花了不少时间踩坑，分享一下自己的安装日志。
一、硬件环境
笔记本：Thinkpad P70 （二手），内存：64G，硬盘：1G固态，win10, 带有8G显存的显卡NVIDIA RTX
二、安装步骤
1、安装 Anaconda软件，用于管理python虚拟环境
官网地址： Anaconda | The World's Most Popular Data Science Platform，下载free版的anaconda，按导航默认安装即可。
参考文章：Anaconda安装和入门(超级简单详细的安装步骤)-CSDN博客
安装后，命令行运行 conda --version
D:\opt\l2>conda --version
conda 23.7.4
2、创建python运行虚拟环境
执行命令：conda create -n l2 python=3.10.12
D:\opt\l2>conda create -n l2 python=3.10.12
...
done
#
# To activate this environment, use
#
# $ conda activate l2
#
# To deactivate an active environment, use
#
# $ conda deactivate
激活虚拟环境: conda activate l2
D:\opt\l2>conda activate l2
(l2) D:\opt\l2>python --version
Python 3.10.12
(l2) D:\opt\l2>
3、安装pytorch
安装pytorch版本是比较麻烦的事情，有一个比较简单的方法：访问url: Start Locally | PyTorch，自动判断当前系统可安装的版本。如下图：
运行推荐的命令安装pytorch，用清华源：
(l2) D:\opt\l2>pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 -i https://pypi.tuna.tsinghua.edu.cn/simple/
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Collecting torch
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fa/47/1a7daf04f40715fc1cdc6f1cc3200228a556d06c843e6ceb58883b745e1b/torch-2.1.0-cp310-cp310-win_amd64.whl (192.3 MB)
---------------------------------------- 192.3/192.3 MB 4.2 MB/s eta 0:00:00
Collecting torchvision
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/59/af/426c2b90f5c4f8aba778746465af9e662680570e950e02379e91c6138285/torchvision-0.16.0-cp310-cp310-win_amd64.whl (1.3 MB)
---------------------------------------- 1.3/1.3 MB 4.5 MB/s eta 0:00:00
Collecting torchaudio
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6b/ba/0e26883fd90f280452bed7edc7906ef9253255f395702751f65fa02afb5c/torchaudio-2.1.0-cp310-cp310-win_amd64.whl (2.3 MB)
---------------------------------------- 2.3/2.3 MB 3.7 MB/s eta 0:00:00
Collecting filelock (from torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/81/54/84d42a0bee35edba99dee7b59a8d4970eccdd44b99fe728ed912106fc781/filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting typing-extensions (from torch)
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/24/21/7d397a4b7934ff4028987914ac1044d3b7d52712f30e2ac7a2ae5bc86dd0/typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Collecting sympy (from torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/05/e6600db80270777c4a64238a98d442f0fd07cc8915be2a1c16da7f2b9e74/sympy-1.12-py3-none-any.whl (5.7 MB)
---------------------------------------- 5.7/5.7 MB 4.8 MB/s eta 0:00:00
Collecting networkx (from torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d5/f0/8fbc882ca80cf077f1b246c0e3c3465f7f415439bdea6b899f6b19f61f70/networkx-3.2.1-py3-none-any.whl (1.6 MB)
---------------------------------------- 1.6/1.6 MB 5.2 MB/s eta 0:00:00
Collecting jinja2 (from torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl (133 kB)
---------------------------------------- 133.1/133.1 kB 2.7 MB/s eta 0:00:00
Collecting fsspec (from torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e8/f6/3eccfb530aac90ad1301c582da228e4763f19e719ac8200752a4841b0b2d/fsspec-2023.10.0-py3-none-any.whl (166 kB)
---------------------------------------- 166.4/166.4 kB 5.0 MB/s eta 0:00:00
Collecting numpy (from torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/57/09/fe9282ffb0217176b0185900945189b6beaec4f94ff46afb76bcd9b68e30/numpy-1.26.1-cp310-cp310-win_amd64.whl (15.8 MB)
---------------------------------------- 15.8/15.8 MB 4.1 MB/s eta 0:00:00
Collecting requests (from torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl (62 kB)
---------------------------------------- 62.6/62.6 kB 1.7 MB/s eta 0:00:00
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2d/7e/18ffce67b6e7637eead295b8a78d293d170d404a633010c3549da9a5e674/Pillow-10.1.0-cp310-cp310-win_amd64.whl (2.6 MB)
---------------------------------------- 2.6/2.6 MB 5.2 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/84/a8/c4aebb8a14a1d39d5135eb8233a0b95831cdc42c4088358449c3ed657044/MarkupSafe-2.1.3-cp310-cp310-win_amd64.whl (17 kB)
Collecting charset-normalizer<4,>=2 (from requests->torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a2/a0/4af29e22cb5942488cf45630cbdd7cefd908768e69bdd90280842e4e8529/charset_normalizer-3.3.2-cp310-cp310-win_amd64.whl (100 kB)
---------------------------------------- 100.3/100.3 kB 2.8 MB/s eta 0:00:00
Collecting idna<4,>=2.5 (from requests->torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fc/34/3030de6f1370931b9dbb4dad48f6ab1015ab1d32447850b9fc94e60097be/idna-3.4-py3-none-any.whl (61 kB)
---------------------------------------- 61.5/61.5 kB 814.3 kB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1 (from requests->torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/b2/b157855192a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be524/urllib3-2.0.7-py3-none-any.whl (124 kB)
---------------------------------------- 124.2/124.2 kB 2.4 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests->torchvision)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl (158 kB)
---------------------------------------- 158.3/158.3 kB 4.6 MB/s eta 0:00:00
Collecting mpmath>=0.19 (from sympy->torch)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl (536 kB)
---------------------------------------- 536.2/536.2 kB 3.7 MB/s eta 0:00:00
Installing collected packages: mpmath, urllib3, typing-extensions, sympy, pillow, numpy, networkx, MarkupSafe, idna, fsspec, filelock, charset-normalizer, certifi, requests, jinja2, torch, torchvision, torchaudio
Successfully installed MarkupSafe-2.1.3 certifi-2023.7.22 charset-normalizer-3.3.2 filelock-3.13.1 fsspec-2023.10.0 idna-3.4 jinja2-3.1.2 mpmath-1.3.0 networkx-3.2.1 numpy-1.26.1 pillow-10.1.0 requests-2.31.0 sympy-1.12 torch-2.1.0 torchaudio-2.1.0 torchvision-0.16.0 typing-extensions-4.8.0 urllib3-2.0.7
验证是否安装成功：
(l2) D:\opt\l2>python
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:34:57) [MSC v.1936 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
False
>>> x = torch.rand(5, 3)
>>> print(x)
tensor([[0.8827, 0.8297, 0.5390],
[0.4590, 0.8473, 0.4074],
[0.4045, 0.0082, 0.4121],
[0.7649, 0.3901, 0.1535],
[0.5408, 0.8168, 0.8615]])
>>>
4、拉取Langchain-Chatchat源代码
有两种方式获取源代码，一种是获取最新代码，一种是获取指定版本的源代码。
# 拉取仓库
git clone https://github.com/chatchat-space/Langchain-Chatchat.git
# 指定版本获取代码
git clone -b v0.2.6 https://github.com/chatchat-space/Langchain-Chatchat.git
(l2) D:\opt\l2>git clone https://github.com/chatchat-space/Langchain-Chatchat.git
Cloning into 'Langchain-Chatchat'...
remote: Enumerating objects: 8144, done.
remote: Counting objects: 100% (902/902), done.
remote: Compressing objects: 100% (450/450), done.
remote: Total 8144 (delta 594), reused 690 (delta 449), pack-reused 7242Receiving objects: 100% (8144/8144), 54.07 MiB |
Resolving deltas: 100% (4900/4900), done.
5、安装依赖包
cd Langchain-Chatchat
# 安装全部依赖 用清华源
pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/
(l2) D:\opt\l2\Langchain-Chatchat>pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Ignoring vllm: markers 'sys_platform == "linux"' don't match your environment
Collecting langchain>=0.0.319 (from -r requirements.txt (line 1))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6a/b6/b8687bf07dfd272a62a9b655de96d9e1e5e9199de1a2dc4f6989ace2e9d5/langchain-0.0.330-py3-none-any.whl (2.0 MB)
---------------------------------------- 2.0/2.0 MB 5.4 MB/s eta 0:00:00
Collecting langchain-experimental>=0.0.30 (from -r requirements.txt (line 2))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/69/71/c7ace3f7b6580c7cad29549cc5d4a5ee7bb14d5c373b8005862c844b92f4/langchain_experimental-0.0.37-py3-none-any.whl (154 kB)
---------------------------------------- 154.9/154.9 kB 9.6 MB/s eta 0:00:00
Collecting fschat==0.2.31 (from fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9c/d1/22b82506f8c0120e25469ce27142ee001ed55da7e917076df278ab777d1e/fschat-0.2.31-py3-none-any.whl (208 kB)
Collecting xformers>=0.0.22.post4 (from -r requirements.txt (line 4))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/71/de/4511904f5e60682b417ac6978cc8393cacb778ab9d9e301d3afd475a3d30/xformers-0.0.22.post7-cp310-cp310-win_amd64.whl (202.1 MB)
---------------------------------------- 202.1/202.1 MB 3.1 MB/s eta 0:00:00
Collecting openai>=0.28.1 (from -r requirements.txt (line 5))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1e/9f/385c25502f437686e4aa715969e5eaf5c2cb5e5ffa7c5cdd52f3c6ae967a/openai-0.28.1-py3-none-any.whl (76 kB)
Collecting sentence_transformers (from -r requirements.txt (line 6))
Using cached sentence_transformers-2.2.2-py3-none-any.whl
Collecting transformers>=4.34 (from -r requirements.txt (line 7))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9a/06/e4ec2a321e57c03b7e9345d709d554a52c33760e5015fdff0919d9459af0/transformers-4.35.0-py3-none-any.whl (7.9 MB)
---------------------------------------- 7.9/7.9 MB 5.9 MB/s eta 0:00:00
Requirement already satisfied: torch>=2.0.1 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from -r requirements.txt (line 8)) (2.1.0)
Requirement already satisfied: torchvision in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from -r requirements.txt (line 9)) (0.16.0)
Requirement already satisfied: torchaudio in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from -r requirements.txt (line 10)) (2.1.0)
Collecting fastapi>=0.104 (from -r requirements.txt (line 11))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f3/4f/0ce34195b63240b6693086496c9bab4ef23999112184399a3e88854c7674/fastapi-0.104.1-py3-none-any.whl (92 kB)
---------------------------------------- 92.9/92.9 kB 2.7 MB/s eta 0:00:00
Collecting nltk>=3.8.1 (from -r requirements.txt (line 12))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/0a/0d20d2c0f16be91b9fa32a77b76c60f9baf6eba419e5ef5deca17af9c582/nltk-3.8.1-py3-none-any.whl (1.5 MB)
Collecting uvicorn~=0.23.1 (from -r requirements.txt (line 13))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/79/96/b0882a1c3f7ef3dd86879e041212ae5b62b4bd352320889231cc735a8e8f/uvicorn-0.23.2-py3-none-any.whl (59 kB)
Collecting starlette~=0.27.0 (from -r requirements.txt (line 14))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/58/f8/e2cca22387965584a409795913b774235752be4176d276714e15e1a58884/starlette-0.27.0-py3-none-any.whl (66 kB)
Collecting pydantic~=1.10.11 (from -r requirements.txt (line 15))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/87/34/d2dc31739fd781daa4581705f0c2ee5fec9daf258aeed6da1ca0149d1da0/pydantic-1.10.13-cp310-cp310-win_amd64.whl (2.1 MB)
Collecting unstructured>=0.10.12 (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/18/47/2a75cf709607f33ad448daaf8a273cb0834c8cb4eaea0819bb17ed926525/unstructured-0.10.28-py3-none-any.whl (1.7 MB)
---------------------------------------- 1.7/1.7 MB 8.9 MB/s eta 0:00:00
Collecting python-magic-bin (from -r requirements.txt (line 17))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/07/c2/094e3d62b906d952537196603a23aec4bcd7c6126bf80eb14e6f9f4be3a2/python_magic_bin-0.4.14-py2.py3-none-win_amd64.whl (409 kB)
Collecting SQLAlchemy==2.0.19 (from -r requirements.txt (line 18))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/71/41/eb74f55dfe9305736430051fc2a77d0479ff80d46aabd832dd583d242ad8/SQLAlchemy-2.0.19-cp310-cp310-win_amd64.whl (2.0 MB)
Collecting faiss-cpu (from -r requirements.txt (line 19))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/0a/3d/d5e431cf7f6c55580581ce965baa56de02be374de838791db9d2a81c20e0/faiss_cpu-1.7.4-cp310-cp310-win_amd64.whl (10.8 MB)
Collecting accelerate (from -r requirements.txt (line 20))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/13/9e/ee987874058f2d93006961f6ff49e0bcb60ab9c26709ebe06bfa8707a4d8/accelerate-0.24.1-py3-none-any.whl (261 kB)
---------------------------------------- 261.4/261.4 kB 5.3 MB/s eta 0:00:00
Collecting spacy (from -r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f5/8f/96f8f02c4719ad48973eb509b2bdaf7afe68447b54cb30f00420032c9c12/spacy-3.7.2-cp310-cp310-win_amd64.whl (12.1 MB)
Collecting PyMuPDF (from -r requirements.txt (line 22))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/80/93/f132b8e4fe503f8cc9b8bf74673c32ce53911cd9fc851d67c3f857e998f0/PyMuPDF-1.23.5-cp310-none-win_amd64.whl (3.5 MB)
---------------------------------------- 3.5/3.5 MB 9.2 MB/s eta 0:00:00
Collecting rapidocr_onnxruntime (from -r requirements.txt (line 23))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/be/1d/46391ab38da2fc3ca5c5b09b1541f2a779ba7f84a65e7067e760b2b422e6/rapidocr_onnxruntime-1.3.8-py3-none-any.whl (14.9 MB)
---------------------------------------- 14.9/14.9 MB 3.5 MB/s eta 0:00:00
Requirement already satisfied: requests in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from -r requirements.txt (line 25)) (2.31.0)
Collecting pathlib (from -r requirements.txt (line 26))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/78/f9/690a8600b93c332de3ab4a344a4ac34f00c8f104917061f779db6a918ed6/pathlib-1.0.1-py3-none-any.whl (14 kB)
Collecting pytest (from -r requirements.txt (line 27))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f3/8c/f16efd81ca8e293b2cc78f111190a79ee539d0d5d36ccd49975cb3beac60/pytest-7.4.3-py3-none-any.whl (325 kB)
---------------------------------------- 325.1/325.1 kB 4.0 MB/s eta 0:00:00
Collecting scikit-learn (from -r requirements.txt (line 28))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/69/8a/cf17d6443f5f537e099be81535a56ab68a473f9393fbffda38cd19899fc8/scikit_learn-1.3.2-cp310-cp310-win_amd64.whl (9.3 MB)
---------------------------------------- 9.3/9.3 MB 4.2 MB/s eta 0:00:00
Collecting numexpr (from -r requirements.txt (line 29))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ce/1d/d0c69195a1e8e913e64096ebcff8c660f1b34414201e8f795add078bcd26/numexpr-2.8.7-cp310-cp310-win_amd64.whl (95 kB)
Collecting numpy~=1.24.4 (from -r requirements.txt (line 42))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/22/55/3d5a7c1142e0d9329ad27cece17933b0e2ab4e54ddc5c1861fbfeb3f7693/numpy-1.24.4-cp310-cp310-win_amd64.whl (14.8 MB)
Collecting pandas~=2.0.3 (from -r requirements.txt (line 43))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ed/30/b97456e7063edac0e5a405128065f0cd2033adfe3716fb2256c186bd41d0/pandas-2.0.3-cp310-cp310-win_amd64.whl (10.7 MB)
Collecting streamlit>=1.26.0 (from -r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/df/e2/e1c8e3abce0e819a4ef3c6a5af4ddbb1eadbca0cf2ebd5e58853edcd58bd/streamlit-1.28.1-py2.py3-none-any.whl (8.4 MB)
---------------------------------------- 8.4/8.4 MB 5.7 MB/s eta 0:00:00
Collecting streamlit-option-menu>=0.3.6 (from -r requirements.txt (line 45))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/5b/7b/6fcab37664539e1a78f12f53002e0a15eb67dd009f540e60f0228e01760e/streamlit_option_menu-0.3.6-py3-none-any.whl (799 kB)
Collecting streamlit-antd-components>=0.1.11 (from -r requirements.txt (line 46))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/73/71/148ff58c35df3f7fad986ae726e965c6fc9584a58724d718bda421bf878b/streamlit_antd_components-0.2.3-py3-none-any.whl (2.4 MB)
Collecting streamlit-chatbox==1.1.10 (from -r requirements.txt (line 47))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/41/d7/6aaee22ec408fc5c3f87023429c63c8b0b97104bceac45e3a726b6c5255b/streamlit_chatbox-1.1.10-py3-none-any.whl (9.6 kB)
Collecting streamlit-aggrid>=0.3.4.post3 (from -r requirements.txt (line 48))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/86/88/dfec35e43a4704c20ce094d7420cd23003bbdf3e8579a20ada63f5f60103/streamlit_aggrid-0.3.4.post3-py3-none-any.whl (3.4 MB)
Collecting httpx~=0.24.1 (from -r requirements.txt (line 49))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ec/91/e41f64f03d2a13aee7e8c819d82ee3aa7cdc484d18c0ae859742597d5aa0/httpx-0.24.1-py3-none-any.whl (75 kB)
Collecting watchdog (from -r requirements.txt (line 50))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9b/6e/ce8d124d03cd3f2941365d9c81d62e3afe43f2dc7e6e86274fa9c2ec2d5b/watchdog-3.0.0-py3-none-win_amd64.whl (82 kB)
Collecting tqdm (from -r requirements.txt (line 51))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/00/e5/f12a80907d0884e6dff9c16d0c0114d81b8cd07dc3ae54c5e962cc83037e/tqdm-4.66.1-py3-none-any.whl (78 kB)
Collecting websockets (from -r requirements.txt (line 52))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/41/b0/5ec054cfcf23adfc88d39359b85e81d043af8a141e3ac8ce40f45a5ce5f4/websockets-12.0-cp310-cp310-win_amd64.whl (124 kB)
---------------------------------------- 125.0/125.0 kB 3.7 MB/s eta 0:00:00
Collecting tiktoken (from -r requirements.txt (line 53))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/7f/92/86ca65b96c80b5f678abc5f76fe167446e9bdf309206a1c4269be4bcaf31/tiktoken-0.5.1-cp310-cp310-win_amd64.whl (759 kB)
Collecting einops (from -r requirements.txt (line 54))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/29/0b/2d1c0ebfd092e25935b86509a9a817159212d82aa43d7fb07eca4eeff2c2/einops-0.7.0-py3-none-any.whl (44 kB)
Collecting scipy (from -r requirements.txt (line 55))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/3a/98/3041681b3a0fef0f59033861a05d846aa1d284b07b531ce35b1b97cc2e93/scipy-1.11.3-cp310-cp310-win_amd64.whl (44.1 MB)
Collecting transformers_stream_generator==0.0.4 (from -r requirements.txt (line 56))
Using cached transformers_stream_generator-0.0.4-py3-none-any.whl
Collecting aiohttp (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1d/60/63b8a1e05956ff92916128c192c9317c8eaaaffacc8fef4ef95db0f84264/aiohttp-3.8.6-cp310-cp310-win_amd64.whl (325 kB)
Collecting markdown2[all] (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f1/98/61276a753f078dd2f3171c9a69fd3f451d220e806b2b1cdca41b8e368b0f/markdown2-2.4.10-py2.py3-none-any.whl (39 kB)
Collecting nh3 (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/53/4c/c34a0a48e81f3328e3300038d52195a2387d82570b0f17df53033c40bc23/nh3-0.2.14-cp37-abi3-win_amd64.whl (574 kB)
Collecting prompt-toolkit>=3.0.0 (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a9/b4/ba77c84edf499877317225d7b7bc047a81f7c2eed9628eeb6bab0ac2e6c9/prompt_toolkit-3.0.39-py3-none-any.whl (385 kB)
Collecting rich>=10.0.0 (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/be/2a/4e62ff633612f746f88618852a626bbe24226eba5e7ac90e91dcfd6a414e/rich-13.6.0-py3-none-any.whl (239 kB)
Collecting shortuuid (from fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/46/644a4df3061e96ef24998c0623d3b12287090ab9a0e0d6ad8408f7b87283/shortuuid-1.0.11-py3-none-any.whl (10 kB)
Requirement already satisfied: typing-extensions>=4.2.0 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from SQLAlchemy==2.0.19->-r requirements.txt (line 18)) (4.8.0)
Collecting greenlet!=0.4.17 (from SQLAlchemy==2.0.19->-r requirements.txt (line 18))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/02/72/36fb2c35547fdf473629579fc35d9a2034592ea3f01710702d81ef596e16/greenlet-3.0.1-cp310-cp310-win_amd64.whl (287 kB)
---------------------------------------- 287.9/287.9 kB 4.5 MB/s eta 0:00:00
Collecting simplejson (from streamlit-chatbox==1.1.10->-r requirements.txt (line 47))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/7f/60/b0071b60e9318bb5c3716e5c0d506d61350670b7fba9a8532ebf3959b88a/simplejson-3.19.2-cp310-cp310-win_amd64.whl (75 kB)
WARNING: fschat 0.2.31 does not provide the extra 'model-worker'
Collecting peft (from fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d1/f5/0380adf697b726e91823bcdd5760a290d7bfac3fbb0f67d30be8ea2df858/peft-0.6.0-py3-none-any.whl (134 kB)
---------------------------------------- 134.9/134.9 kB 3.9 MB/s eta 0:00:00
Collecting sentencepiece (from fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/21/8c/54bd0e155ec945cd8bcf387c25b2d337b345f2e56939336d71183dc1804f/sentencepiece-0.1.99-cp310-cp310-win_amd64.whl (977 kB)
Collecting protobuf (from fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/72/eb/a07cc36ed9b119062886935a8371da8a977dda30b19cf57fe9162589b6f7/protobuf-4.25.0-cp310-abi3-win_amd64.whl (413 kB)
---------------------------------------- 413.2/413.2 kB 3.7 MB/s eta 0:00:00
Collecting PyYAML>=5.3 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/24/97/9b59b43431f98d01806b288532da38099cc6f2fea0f3d712e21e269c0279/PyYAML-6.0.1-cp310-cp310-win_amd64.whl (145 kB)
Collecting anyio<4.0 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/19/24/44299477fe7dcc9cb58d0a57d5a7588d6af2ff403fdd2d47a246c91a3246/anyio-3.7.1-py3-none-any.whl (80 kB)
Collecting async-timeout<5.0.0,>=4.0.0 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a7/fa/e01228c2938de91d47b307831c62ab9e4001e747789d0b05baf779a6488c/async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Collecting dataclasses-json<0.7,>=0.5.7 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/21/1f/1cff009cff64420572b9f75b70e4a054095719179a172297dfdd65843162/dataclasses_json-0.6.1-py3-none-any.whl (27 kB)
Collecting jsonpatch<2.0,>=1.33 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/73/07/02e16ed01e04a374e644b575638ec7987ae846d25ad97bcc9945a3ee4b0e/jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Collecting langsmith<0.1.0,>=0.0.52 (from langchain>=0.0.319->-r requirements.txt (line 1))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/08/e8/8b1b439e8b5e26c0d4faf1c121d8b05af9ee4b62c9d34c18e930efff8ece/langsmith-0.0.57-py3-none-any.whl (44 kB)
---------------------------------------- 44.5/44.5 kB 1.1 MB/s eta 0:00:00
Collecting tenacity<9.0.0,>=8.1.0 (from langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f4/f1/990741d5bb2487d529d20a433210ffa136a367751e454214013b441c4575/tenacity-8.2.3-py3-none-any.whl (24 kB)
Requirement already satisfied: filelock in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torch>=2.0.1->-r requirements.txt (line 8)) (3.13.1)
Requirement already satisfied: sympy in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torch>=2.0.1->-r requirements.txt (line 8)) (1.12)
Requirement already satisfied: networkx in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torch>=2.0.1->-r requirements.txt (line 8)) (3.2.1)
Requirement already satisfied: jinja2 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torch>=2.0.1->-r requirements.txt (line 8)) (3.1.2)
Requirement already satisfied: fsspec in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torch>=2.0.1->-r requirements.txt (line 8)) (2023.10.0)
Collecting huggingface-hub>=0.4.0 (from sentence_transformers->-r requirements.txt (line 6))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ef/b5/b6107bd65fa4c96fdf00e4733e2fe5729bb9e5e09997f63074bb43d3ab28/huggingface_hub-0.18.0-py3-none-any.whl (301 kB)
Collecting packaging>=20.0 (from transformers>=4.34->-r requirements.txt (line 7))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl (53 kB)
Collecting regex!=2019.12.17 (from transformers>=4.34->-r requirements.txt (line 7))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/33/03/91c9509b43154795fb848a4cf8cef5b37302b3b3ccf8a9763046ea528c6b/regex-2023.10.3-cp310-cp310-win_amd64.whl (269 kB)
Collecting tokenizers<0.15,>=0.14 (from transformers>=4.34->-r requirements.txt (line 7))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/92/02/15556b80450301d2ef014bc598df4352bfb39631c5fcff758d8e0ac9f065/tokenizers-0.14.1-cp310-none-win_amd64.whl (2.2 MB)
Collecting safetensors>=0.3.1 (from transformers>=4.34->-r requirements.txt (line 7))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/35/c0/d5c1ef78fd9d52913b70574501562c38916927f1778a6baf4f0b80129463/safetensors-0.4.0-cp310-none-win_amd64.whl (277 kB)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from torchvision->-r requirements.txt (line 9)) (10.1.0)
Collecting click (from nltk>=3.8.1->-r requirements.txt (line 12))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl (97 kB)
Collecting joblib (from nltk>=3.8.1->-r requirements.txt (line 12))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/10/40/d551139c85db202f1f384ba8bcf96aca2f329440a844f924c8a0040b6d02/joblib-1.3.2-py3-none-any.whl (302 kB)
Collecting h11>=0.8 (from uvicorn~=0.23.1->-r requirements.txt (line 13))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl (58 kB)
Collecting chardet (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/38/6f/f5fbc992a329ee4e0f288c1fe0e2ad9485ed064cac731ed2fe47dcc38cbf/chardet-5.2.0-py3-none-any.whl (199 kB)
Collecting filetype (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/18/79/1b8fa1bb3568781e84c9200f951c735f3f157429f44be0495da55894d620/filetype-1.2.0-py2.py3-none-any.whl (19 kB)
Collecting python-magic (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6c/73/9f872cb81fc5c3bb48f7227872c28975f998f3e7c2b1c16e95e6432bbb90/python_magic-0.4.27-py2.py3-none-any.whl (13 kB)
Collecting lxml (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/50/ba/cb7bc9728a3be4e00dfd658fc76dc64fd9dbc3d5492ff44cda70574329c6/lxml-4.9.3-cp310-cp310-win_amd64.whl (3.8 MB)
Collecting tabulate (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/40/44/4a5f08c96eb108af5cb50b41f76142f0afa346dfa99d5296fe7202a11854/tabulate-0.9.0-py3-none-any.whl (35 kB)
Collecting beautifulsoup4 (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/57/f4/a69c20ee4f660081a7dedb1ac57f29be9378e04edfcb90c526b923d4bebc/beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)
Collecting emoji (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/96/c6/0114b2040a96561fd1b44c75df749bbd3c898bf8047fb5ce8d7590d2dee6/emoji-2.8.0-py2.py3-none-any.whl (358 kB)
Collecting python-iso639 (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b8/6d/5d1f7e5c1b0c58b700eb67dbb570f9381afc90bc0535686a89e90eac5dfb/python_iso639-2023.6.15-py3-none-any.whl (275 kB)
Collecting langdetect (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached langdetect-1.0.9-py3-none-any.whl
Collecting rapidfuzz (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/de/8f/404add97fef8c94bb41d6d1be08c5f8430f48d280fd3cee979cceab94c46/rapidfuzz-3.5.2-cp310-cp310-win_amd64.whl (1.7 MB)
---------------------------------------- 1.7/1.7 MB 9.9 MB/s eta 0:00:00
Collecting backoff (from unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/df/73/b6e24bd22e6720ca8ee9a85a0c4a2971af8497d8f3193fa05390cbd46e09/backoff-2.2.1-py3-none-any.whl (15 kB)
Collecting psutil (from accelerate->-r requirements.txt (line 20))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c5/b2/699c50fe0b0402a1ccb64ad71313bcb740e735008dd3ab9abeddbe148e45/psutil-5.9.6-cp37-abi3-win_amd64.whl (252 kB)
Collecting spacy-legacy<3.1.0,>=3.0.11 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/55/12e842c70ff8828e34e543a2c7176dac4da006ca6901c9e8b43efab8bc6b/spacy_legacy-3.0.12-py2.py3-none-any.whl (29 kB)
Collecting spacy-loggers<2.0.0,>=1.0.0 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/33/78/d1a1a026ef3af911159398c939b1509d5c36fe524c7b644f34a5146c4e16/spacy_loggers-1.0.5-py3-none-any.whl (22 kB)
Collecting murmurhash<1.1.0,>=0.28.0 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ed/9d/d62d12e3ecc6f99eddea6289413669a905d2ebb15cf9fe075336ca6cceaa/murmurhash-1.0.10-cp310-cp310-win_amd64.whl (25 kB)
Collecting cymem<2.1.0,>=2.0.2 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/51/12/4aa9eec680c6d12b2275d479e159c3d063d7c757175063dd45386e15b39d/cymem-2.0.8-cp310-cp310-win_amd64.whl (39 kB)
Collecting preshed<3.1.0,>=3.0.2 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f3/72/108426ca3b6e7f16db30b3b9396e3fa45a3fd5a76f6532ab04beada2e4e3/preshed-3.0.9-cp310-cp310-win_amd64.whl (122 kB)
Collecting thinc<8.3.0,>=8.1.8 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/b4/d3652585e9819d9cc197e4287112caaea388caa5819640bfe679291665d0/thinc-8.2.1-cp310-cp310-win_amd64.whl (1.5 MB)
Collecting wasabi<1.2.0,>=0.9.1 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/8f/69/26cbf0bad11703241cb84d5324d868097f7a8faf2f1888354dac8883f3fc/wasabi-1.1.2-py3-none-any.whl (27 kB)
Collecting srsly<3.0.0,>=2.4.3 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/0a/ed/d2c37221fe1975f4b6e8e3cf200d25b905b77e18f6a660b3dc149ade6192/srsly-2.4.8-cp310-cp310-win_amd64.whl (481 kB)
Collecting catalogue<2.1.0,>=2.0.6 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9e/96/d32b941a501ab566a16358d68b6eb4e4acc373fab3c3c4d7d9e649f7b4bb/catalogue-2.0.10-py3-none-any.whl (17 kB)
Collecting weasel<0.4.0,>=0.1.0 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/76/41433e70b7c9ed47db8206536ecc6408362db44111c27f59e76af20377f2/weasel-0.3.3-py3-none-any.whl (49 kB)
Collecting typer<0.10.0,>=0.3.0 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/bf/0e/c68adf10adda05f28a6ed7b9f4cd7b8e07f641b44af88ba72d9c89e4de7a/typer-0.9.0-py3-none-any.whl (45 kB)
Collecting smart-open<7.0.0,>=5.2.1 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/fc/d9/d97f1db64b09278aba64e8c81b5d322d436132df5741c518f3823824fae0/smart_open-6.4.0-py3-none-any.whl (57 kB)
Requirement already satisfied: setuptools in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from spacy->-r requirements.txt (line 21)) (68.2.2)
Collecting langcodes<4.0.0,>=3.2.0 (from spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/fe/c3/0d04d248624a181e57c2870127dfa8d371973561caf54333c85e8f9133a2/langcodes-3.3.0-py3-none-any.whl (181 kB)
Collecting PyMuPDFb==1.23.5 (from PyMuPDF->-r requirements.txt (line 22))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/cb/76/4eade1b24e1241431b35d6f955afdff2595bafc5a5da8f37fbe1059f989d/PyMuPDFb-1.23.5-py3-none-win_amd64.whl (24.4 MB)
---------------------------------------- 24.4/24.4 MB 3.4 MB/s eta 0:00:00
Collecting pyclipper>=1.2.0 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/60/61/354f484ab7969a601327646bbaeb1b799508b4e81946ea4d52bbf9d779c6/pyclipper-1.3.0.post5-cp310-cp310-win_amd64.whl (108 kB)
Collecting onnxruntime>=1.7.0 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/97/26/c5b85935d80102e874fdd051a69bcedac15b6a9695efcafee4d2e1e690e3/onnxruntime-1.16.1-cp310-cp310-win_amd64.whl (7.2 MB)
Collecting opencv-python>=4.5.1.48 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/38/d2/3e8c13ffc37ca5ebc6f382b242b44acb43eb489042e1728407ac3904e72f/opencv_python-4.8.1.78-cp37-abi3-win_amd64.whl (38.1 MB)
Collecting six>=1.15.0 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting Shapely>=1.7.1 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/28/32/c404f6d5566025b9d94eef0cc21039b0a4c10a392bad2f81edf778b3c2bc/shapely-2.0.2-cp310-cp310-win_amd64.whl (1.4 MB)
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision->-r requirements.txt (line 9))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/3e/14/0030e542f2acfea43635e55584c114e6cfd94d342393a5f71f74c172dc35/Pillow-9.5.0-cp310-cp310-win_amd64.whl (2.5 MB)
---------------------------------------- 2.5/2.5 MB 5.2 MB/s eta 0:00:00
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from requests->-r requirements.txt (line 25)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from requests->-r requirements.txt (line 25)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from requests->-r requirements.txt (line 25)) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from requests->-r requirements.txt (line 25)) (2023.7.22)
Collecting iniconfig (from pytest->-r requirements.txt (line 27))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ef/a6/62565a6e1cf69e10f5727360368e451d4b7f58beeac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting pluggy<2.0,>=0.12 (from pytest->-r requirements.txt (line 27))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/05/b8/42ed91898d4784546c5f06c60506400548db3f7a4b3fb441cba4e5c17952/pluggy-1.3.0-py3-none-any.whl (18 kB)
Collecting exceptiongroup>=1.0.0rc8 (from pytest->-r requirements.txt (line 27))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Collecting tomli>=1.0.0 (from pytest->-r requirements.txt (line 27))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting colorama (from pytest->-r requirements.txt (line 27))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn->-r requirements.txt (line 28))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Collecting python-dateutil>=2.8.2 (from pandas~=2.0.3->-r requirements.txt (line 43))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2020.1 (from pandas~=2.0.3->-r requirements.txt (line 43))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/32/4d/aaf7eff5deb402fd9a24a1449a8119f00d74ae9c2efa79f8ef9994261fc2/pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Collecting tzdata>=2022.1 (from pandas~=2.0.3->-r requirements.txt (line 43))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d5/fb/a79efcab32b8a1f1ddca7f35109a50e4a80d42ac1c9187ab46522b2407d7/tzdata-2023.3-py2.py3-none-any.whl (341 kB)
Collecting altair<6,>=4.0 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/17/16/b12fca347ff9d062e3c44ad9641d2ec50364570a059f3078ada3a5119d7a/altair-5.1.2-py3-none-any.whl (516 kB)
Collecting blinker<2,>=1.0.0 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fa/2a/7f3714cbc6356a0efec525ce7a0613d581072ed6eb53eb7b9754f33db807/blinker-1.7.0-py3-none-any.whl (13 kB)
Collecting cachetools<6,>=4.0 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a2/91/2d843adb9fbd911e0da45fbf6f18ca89d07a087c3daa23e955584f90ebf4/cachetools-5.3.2-py3-none-any.whl (9.3 kB)
Collecting importlib-metadata<7,>=1.4 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/cc/37/db7ba97e676af155f5fcb1a35466f446eadc9104e25b83366e8088c9c926/importlib_metadata-6.8.0-py3-none-any.whl (22 kB)
Collecting pyarrow>=6.0 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/80/ac/3266ac391c5b0a4da70be7a96fb63fdba7bbc5620ecc3f9e5a30262259f7/pyarrow-14.0.0-cp310-cp310-win_amd64.whl (24.6 MB)
---------------------------------------- 24.6/24.6 MB 3.5 MB/s eta 0:00:00
Collecting toml<2,>=0.10.1 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting tzlocal<6,>=1.1 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/97/3f/c4c51c55ff8487f2e6d0e618dba917e3c3ee2caae6cf0fbb59c9b1876f2e/tzlocal-5.2-py3-none-any.whl (17 kB)
Collecting validators<1,>=0.2 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/3a/0c/785d317eea99c3739821718f118c70537639aa43f96bfa1d83a71f68eaf6/validators-0.22.0-py3-none-any.whl (26 kB)
Collecting gitpython!=3.1.19,<4,>=3.0.7 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/8d/c4/82b858fb6483dfb5e338123c154d19c043305b01726a67d89532b8f8f01b/GitPython-3.1.40-py3-none-any.whl (190 kB)
Collecting pydeck<1,>=0.8.0b4 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/10/4b/2fc80540e2d3903452245bb657c7f758ec7342420507d1e4091b0161856e/pydeck-0.8.1b0-py2.py3-none-any.whl (4.8 MB)
Collecting tornado<7,>=6.0.3 (from streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/19/07/65898bfa51d1a901f7798c36b3cf7c8d1df0c31a7178b79f75edf6d038cd/tornado-6.3.3-cp38-abi3-win_amd64.whl (429 kB)
Collecting python-decouple<4.0,>=3.6 (from streamlit-aggrid>=0.3.4.post3->-r requirements.txt (line 48))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a2/d4/9193206c4563ec771faf2ccf54815ca7918529fe81f6adb22ee6d0e06622/python_decouple-3.8-py3-none-any.whl (9.9 kB)
Collecting httpcore<0.18.0,>=0.15.0 (from httpx~=0.24.1->-r requirements.txt (line 49))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/94/2c/2bde7ff8dd2064395555220cbf7cba79991172bf5315a07eb3ac7688d9f1/httpcore-0.17.3-py3-none-any.whl (74 kB)
Collecting sniffio (from httpx~=0.24.1->-r requirements.txt (line 49))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/a0/5dba8ed157b0136607c7f2151db695885606968d1fae123dc3391e0cfdbf/sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting unstructured.pytesseract>=0.3.12 (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c5/83/4554641f47672fe915be03101cf1c41ab8a3d373518b3240deb8e3a9e7ac/unstructured.pytesseract-0.3.12-py3-none-any.whl (14 kB)
Collecting xlrd (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/0c/c2a72d51fe56e08a08acc85d13013558a2d793028ae7385448a6ccdfae64/xlrd-2.0.1-py2.py3-none-any.whl (96 kB)
Collecting pdfminer.six (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/46/68/b3fb5f073bcd3df9143a3520289c147351bfa3c1b096d44081f38fd1c247/pdfminer.six-20221105-py3-none-any.whl (5.6 MB)
Collecting unstructured-inference==0.7.10 (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4d/74/2dec2b52277b481c9c8297e4baec11871a8572f3d00dbda501f30397ac28/unstructured_inference-0.7.10-py3-none-any.whl (61 kB)
---------------------------------------- 61.8/61.8 kB 3.4 MB/s eta 0:00:00
Collecting python-pptx<=0.6.21 (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached python_pptx-0.6.21-py3-none-any.whl
Collecting pypandoc (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/01/35/d39a3abc5b0e5ca646081c492b37dc737e0d4563689a78213680de354fc6/pypandoc-1.12-py3-none-any.whl (20 kB)
Collecting python-docx>=1.0.1 (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/5f/d8/6948f7ac00edf74bfa52b3c5e3073df20284bec1db466d13e668fe991707/python_docx-1.1.0-py3-none-any.whl (239 kB)
---------------------------------------- 239.6/239.6 kB 4.9 MB/s eta 0:00:00
Collecting msg-parser (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/15/8b/5738b32acc6acdf92d04d5e691bf70a379e78264e55843542e1888d4a10e/msg_parser-1.2.0-py2.py3-none-any.whl (101 kB)
Collecting openpyxl (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6a/94/a59521de836ef0da54aaf50da6c4da8fb4072fb3053fa71f052fd9399e7a/openpyxl-3.1.2-py2.py3-none-any.whl (249 kB)
Collecting pdf2image (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e2/8c/37a7467d88c47e435429c147020c2fb53206fabfc44b8b72f9728e79ce1a/pdf2image-1.16.3-py3-none-any.whl (11 kB)
Collecting markdown (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/70/58/2c5a654173937d9f540a4971c569b44dcd55e5424a484d954cdaeebcf79c/Markdown-3.5.1-py3-none-any.whl (102 kB)
---------------------------------------- 102.2/102.2 kB 3.0 MB/s eta 0:00:00
Collecting onnx (from unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1b/43/6e84bf35a9201fb344b1a98edf7095c4fa1cf1478dfb6638d7b77f5475e6/onnx-1.15.0-cp310-cp310-win_amd64.whl (14.3 MB)
---------------------------------------- 14.3/14.3 MB 3.7 MB/s eta 0:00:00
Collecting layoutparser[layoutmodels,tesseract] (from unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/08/cf/0bfbea1b2ace91af45e15bdec885e05992dc9150907a8398b3d305eddfd2/layoutparser-0.3.4-py3-none-any.whl (19.2 MB)
Collecting python-multipart (from unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b4/ff/b1e11d8bffb5e0e1b6d27f402eeedbeb9be6df2cdbc09356a1ae49806dbf/python_multipart-0.0.6-py3-none-any.whl (45 kB)
Collecting onnxruntime>=1.7.0 (from rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/aa/2a/9d7edd7a329a7f34e25af60a1e47468cbc864766898c7b9d3b540d0f1539/onnxruntime-1.15.1-cp310-cp310-win_amd64.whl (6.7 MB)
Collecting attrs>=17.3.0 (from aiohttp->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f0/eb/fcb708c7bf5056045e9e98f62b93bd7467eb718b0202e7698eb11d66416c/attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/24/d1/56b6d5eb964161c55a8a7ad53fe4c93a694e44d04fd1405f3c1b98de5627/multidict-6.0.4-cp310-cp310-win_amd64.whl (28 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/98/21/9ef4adf36cfac771518c3bf687bc9b92451bdaf01ec770879f19e7e5b3c7/yarl-1.9.2-cp310-cp310-win_amd64.whl (61 kB)
Collecting frozenlist>=1.1.1 (from aiohttp->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ad/1a/0035d6dd5e51580c060f80ac6819f0ea55da2b1f2dad8ad15f5d3e50d91c/frozenlist-1.4.0-cp310-cp310-win_amd64.whl (44 kB)
Collecting aiosignal>=1.1.2 (from aiohttp->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/76/ac/a7305707cb852b7e16ff80eaf5692309bde30e2b1100a1fcacdc8f731d97/aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting jsonschema>=3.0 (from altair<6,>=4.0->streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ce/aa/d1bd0b5ec568a903cc3ebcb6b096ab65c1d971c8a01ca3bf3cf788c3c646/jsonschema-4.19.2-py3-none-any.whl (83 kB)
---------------------------------------- 83.6/83.6 kB 4.9 MB/s eta 0:00:00
Collecting toolz (from altair<6,>=4.0->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/7f/5c/922a3508f5bda2892be3df86c74f9cf1e01217c2b1f8a0ac4841d903e3e9/toolz-0.12.0-py3-none-any.whl (55 kB)
Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ed/3c/cebfdcad015240014ff08b883d1c0c427f2ba45ae8c6572851b6ef136cad/marshmallow-3.20.1-py3-none-any.whl (49 kB)
Collecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/65/f3/107a22063bf27bdccf2024833d3445f4eea42b2e598abfbd46f6a63b6cb0/typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3.0.7->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/fd/5b/8f0c4a5bb9fd491c277c21eff7ccae71b47d43c4446c9d0c6cff2fe8c2c4/gitdb-4.0.11-py3-none-any.whl (62 kB)
Collecting zipp>=0.5 (from importlib-metadata<7,>=1.4->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/66/48866fc6b158c81cc2bfecc04c480f105c6040e8b077bc54c634b4a67926/zipp-3.17.0-py3-none-any.whl (7.4 kB)
Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/12/f6/0232cc0c617e195f06f810534d00b74d2f348fe71b2118009ad8ad31f878/jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)
Collecting coloredlogs (from onnxruntime>=1.7.0->rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a7/06/3d6badcf13db419e25b07041d9c7b4a2c331d3f4e7134445ec5df57714cd/coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Collecting flatbuffers (from onnxruntime>=1.7.0->rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Collecting wcwidth (from prompt-toolkit>=3.0.0->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/19/0b/00728863778b14ececfc97e40850fd71529b6a1695907981cc3fdc085ba6/wcwidth-0.2.9-py2.py3-none-any.whl (102 kB)
---------------------------------------- 102.3/102.3 kB 3.0 MB/s eta 0:00:00
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from jinja2->torch>=2.0.1->-r requirements.txt (line 8)) (2.1.3)
Collecting XlsxWriter>=0.5.7 (from python-pptx<=0.6.21->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f7/3e/05ba2194cd5073602422859c949a4f21310a3c49bf8dccde9e03d4522b11/XlsxWriter-3.1.9-py3-none-any.whl (154 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=10.0.0->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=10.0.0->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/43/88/29adf0b44ba6ac85045e63734ae0997d3c58d8b1a91c914d240828d0d73d/Pygments-2.16.1-py3-none-any.whl (1.2 MB)
Collecting blis<0.8.0,>=0.7.8 (from thinc<8.3.0,>=8.1.8->spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ad/65/d9fd07e11499e0a3162c6d61ae430172125e5c340c89c40504189d5299b9/blis-0.7.11-cp310-cp310-win_amd64.whl (6.6 MB)
Collecting confection<1.0.0,>=0.0.1 (from thinc<8.3.0,>=8.1.8->spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/93/f8/e89268a1f885048fb2ee6b5c9f93c4e90de768534acfef3652f87d97d4cb/confection-0.1.3-py3-none-any.whl (34 kB)
Collecting huggingface-hub>=0.4.0 (from sentence_transformers->-r requirements.txt (line 6))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/aa/f3/3fc97336a0e90516901befd4f500f08d691034d387406fdbde85bea827cc/huggingface_hub-0.17.3-py3-none-any.whl (295 kB)
Collecting cloudpathlib<0.17.0,>=0.7.0 (from weasel<0.4.0,>=0.1.0->spacy->-r requirements.txt (line 21))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/0f/6e/45b57a7d4573d85d0b0a39d99673dc1f5eea9d92a1a4603b35e968fbf89a/cloudpathlib-0.16.0-py3-none-any.whl (45 kB)
Collecting soupsieve>1.2 (from beautifulsoup4->unstructured>=0.10.12->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/4c/f3/038b302fdfbe3be7da016777069f26ceefe11a681055ea1f7817546508e3/soupsieve-2.5-py3-none-any.whl (36 kB)
Collecting wavedrom (from markdown2[all]->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached wavedrom-2.0.3.post3-py2.py3-none-any.whl
Collecting olefile>=0.46 (from msg-parser->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached olefile-0.46-py2.py3-none-any.whl
Collecting et-xmlfile (from openpyxl->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/96/c2/3dd434b0108730014f1b96fd286040dc3bcb70066346f7e01ec2ac95865f/et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)
Collecting cryptography>=36.0.0 (from pdfminer.six->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/86/35/f03a42444866ef7f23134812a05012dcb509418214fb78ec848f28cd14b8/cryptography-41.0.5-cp37-abi3-win_amd64.whl (2.7 MB)
---------------------------------------- 2.7/2.7 MB 2.3 MB/s eta 0:00:00
Requirement already satisfied: mpmath>=0.19 in c:\users\chenjd\.conda\envs\l2\lib\site-packages (from sympy->torch>=2.0.1->-r requirements.txt (line 8)) (1.3.0)
Collecting cffi>=1.12 (from cryptography>=36.0.0->pdfminer.six->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/be/3e/0b197d1bfbf386a90786b251dbf2634a15f2ea3d4e4070e99c7d1c7689cf/cffi-1.16.0-cp310-cp310-win_amd64.whl (181 kB)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a7/a5/10f97f73544edcdef54409f1d839f6049a0d79df68adbc1ceb24d1aaca42/smmap-5.0.1-py3-none-any.whl (24 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1c/24/83349ac2189cc2435e84da3f69ba3c97314d3c0622628e55171c6798ed80/jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Collecting referencing>=0.28.4 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.26.0->-r requirements.txt (line 44))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/be/8e/56d6f1e2d591f4d6cbcba446cac4a1b0dc4f584537e2071d9bcee8eeab6b/referencing-0.30.2-py3-none-any.whl (25 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.26.0->-r requirements.txt (line 44))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6c/51/9fe0a5cc71ec86abfce04b14c778e99c0db2152c67b855c972dc52ce2e54/rpds_py-0.12.0-cp310-none-win_amd64.whl (186 kB)
---------------------------------------- 186.4/186.4 kB 3.7 MB/s eta 0:00:00
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=10.0.0->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain>=0.0.319->-r requirements.txt (line 1))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/2a/e2/5d3f6ada4297caebe1a2add3b126fe800c96f56dbe5d1988a2cbe0b267aa/mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.7.0->rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f0/0f/310fb31e39e2d734ccaa2c0fb981ee41f7bd5056ce9bc29b2248bd569169/humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Collecting iopath (from layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached iopath-0.1.10-py3-none-any.whl
Collecting pdfplumber (from layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f8/d3/f58c2d5d86a585e438c6708f568eca79e7c4e6ee3d5210cf8b31d38cb021/pdfplumber-0.10.3-py3-none-any.whl (48 kB)
---------------------------------------- 49.0/49.0 kB 1.2 MB/s eta 0:00:00
Collecting pytesseract (from layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c5/54/ec007336f38d2d4ce61f3544af3e6855dacbf04a1ac8294f10cabe81146f/pytesseract-0.3.10-py3-none-any.whl (14 kB)
Collecting effdet (from layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9c/13/563119fe0af82aca5a3b89399c435953072c39515c2e818eb82793955c3b/effdet-0.4.1-py3-none-any.whl (112 kB)
Collecting svgwrite (from wavedrom->markdown2[all]->fschat==0.2.31->fschat[model_worker]==0.2.31->-r requirements.txt (line 3))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/84/15/640e399579024a6875918839454025bb1d5f850bb70d96a11eabb644d11c/svgwrite-1.4.3-py3-none-any.whl (67 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/62/d5/5f610ebe421e85889f2e55e33b7f9a6795bd982198517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting pyreadline3 (from humanfriendly>=9.1->coloredlogs->onnxruntime>=1.7.0->rapidocr_onnxruntime->-r requirements.txt (line 23))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/56/fc/a3c13ded7b3057680c8ae95a9b6cc83e63657c38e0005c400a5d018a33a7/pyreadline3-3.4.1-py3-none-any.whl (95 kB)
Collecting timm>=0.9.2 (from effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f0/1e/05287cb8984229d101874433df472b1fa3dcd6f746ccb6e8a26c7deeb1c7/timm-0.9.10-py3-none-any.whl (2.2 MB)
---------------------------------------- 2.2/2.2 MB 4.7 MB/s eta 0:00:00
Collecting pycocotools>=2.0.2 (from effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6e/03/66168e1940ad0ea745cc6489fd1bd9b8c296b20a7c0102bd329382880659/pycocotools-2.0.7-cp310-cp310-win_amd64.whl (84 kB)
Collecting omegaconf>=2.0 (from effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e3/94/1843518e420fa3ed6919835845df698c7e27e183cb997394e4a670973a65/omegaconf-2.3.0-py3-none-any.whl (79 kB)
Collecting portalocker (from iopath->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl (17 kB)
Collecting pypdfium2>=4.18.0 (from pdfplumber->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/87/8f/3bb2dc52454e6fae5b8b51672d4739576bb4c0a82dab998bc2c8881a7b79/pypdfium2-4.23.1-py3-none-win_amd64.whl (2.7 MB)
---------------------------------------- 2.7/2.7 MB 5.3 MB/s eta 0:00:00
Collecting antlr4-python3-runtime==4.9.* (from omegaconf>=2.0->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached antlr4_python3_runtime-4.9.3-py3-none-any.whl
Collecting matplotlib>=2.1.0 (from pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/96/96/117ba44070780fea0874279e5bf84816d2b62c3b151b6fcfc738022f55f4/matplotlib-3.8.1-cp310-cp310-win_amd64.whl (7.6 MB)
---------------------------------------- 7.6/7.6 MB 6.2 MB/s eta 0:00:00
Collecting pywin32>=226 (from portalocker->iopath->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d3/d6/891894edec688e72c2e308b3243fad98b4066e1839fd2fe78f04129a9d31/pywin32-306-cp310-cp310-win_amd64.whl (9.2 MB)
Collecting contourpy>=1.0.1 (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fd/7c/168f8343f33d861305e18c56901ef1bb675d3c7f977f435ec72751a71a54/contourpy-1.2.0-cp310-cp310-win_amd64.whl (186 kB)
---------------------------------------- 186.7/186.7 kB 5.5 MB/s eta 0:00:00
Collecting cycler>=0.10 (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl (8.3 kB)
Collecting fonttools>=4.22.0 (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7d/84/12824c89f99ce780ec7687cb7d733af496198f966e05a6dbaa68f305656f/fonttools-4.44.0-cp310-cp310-win_amd64.whl (2.1 MB)
---------------------------------------- 2.1/2.1 MB 7.2 MB/s eta 0:00:00
Collecting kiwisolver>=1.3.1 (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/4a/a1/8a9c9be45c642fa12954855d8b3a02d9fd8551165a558835a19508fec2e6/kiwisolver-1.4.5-cp310-cp310-win_amd64.whl (56 kB)
Collecting pyparsing>=2.3.1 (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->unstructured-inference==0.7.10->unstructured[all-docs]>=0.10.12->-r requirements.txt (line 16))
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl (103 kB)
Installing collected packages: wcwidth, sentencepiece, pywin32, pytz, python-magic-bin, python-decouple, pyreadline3, pyclipper, pathlib, nh3, flatbuffers, filetype, faiss-cpu, cymem, antlr4-python3-runtime, zipp, XlsxWriter, xlrd, websockets, watchdog, validators, tzdata, tornado, toolz, tomli, toml, threadpoolctl, tenacity, tabulate, svgwrite, spacy-loggers, spacy-legacy, soupsieve, sniffio, smmap, smart-open, six, simplejson, shortuuid, safetensors, rpds-py, regex, rapidfuzz, PyYAML, python-multipart, python-magic, python-iso639, pypdfium2, pyparsing, pypandoc, PyMuPDFb, pygments, pydantic, pycparser, psutil, protobuf, prompt-toolkit, portalocker, pluggy, pillow, packaging, olefile, numpy, mypy-extensions, murmurhash, multidict, mdurl, markdown2, markdown, lxml, langcodes, kiwisolver, jsonpointer, joblib, iniconfig, humanfriendly, h11, greenlet, frozenlist, fonttools, exceptiongroup, et-xmlfile, emoji, einops, cycler, colorama, cloudpathlib, chardet, catalogue, cachetools, blinker, backoff, attrs, async-timeout, yarl, wavedrom, wasabi, unstructured.pytesseract, tzlocal, typing-inspect, tqdm, tiktoken, srsly, SQLAlchemy, Shapely, scipy, referencing, python-pptx, python-docx, python-dateutil, pytest, pytesseract, PyMuPDF, pydeck, pyarrow, preshed, pdf2image, openpyxl, opencv-python, onnx, omegaconf, numexpr, msg-parser, marshmallow, markdown-it-py, langsmith, langdetect, jsonpatch, importlib-metadata, gitdb, contourpy, coloredlogs, click, cffi, blis, beautifulsoup4, anyio, aiosignal, xformers, uvicorn, typer, starlette, scikit-learn, rich, pandas, onnxruntime, nltk, matplotlib, jsonschema-specifications, iopath, huggingface-hub, httpcore, gitpython, dataclasses-json, cryptography, confection, aiohttp, weasel, unstructured, tokenizers, timm, thinc, rapidocr_onnxruntime, pycocotools, pdfminer.six, openai, langchain, jsonschema, httpx, fastapi, accelerate, transformers, spacy, pdfplumber, langchain-experimental, fschat, effdet, altair, transformers_stream_generator, streamlit, sentence_transformers, peft, layoutparser, streamlit-option-menu, streamlit-chatbox, streamlit-antd-components, streamlit-aggrid, unstructured-inference
Attempting uninstall: pillow
Found existing installation: Pillow 10.1.0
Uninstalling Pillow-10.1.0:
Successfully uninstalled Pillow-10.1.0
Attempting uninstall: numpy
Found existing installation: numpy 1.26.1
Uninstalling numpy-1.26.1:
Successfully uninstalled numpy-1.26.1
Successfully installed PyMuPDF-1.23.5 PyMuPDFb-1.23.5 PyYAML-6.0.1 SQLAlchemy-2.0.19 Shapely-2.0.2 XlsxWriter-3.1.9 accelerate-0.24.1 aiohttp-3.8.6 aiosignal-1.3.1 altair-5.1.2 antlr4-python3-runtime-4.9.3 anyio-3.7.1 async-timeout-4.0.3 attrs-23.1.0 backoff-2.2.1 beautifulsoup4-4.12.2 blinker-1.7.0 blis-0.7.11 cachetools-5.3.2 catalogue-2.0.10 cffi-1.16.0 chardet-5.2.0 click-8.1.7 cloudpathlib-0.16.0 colorama-0.4.6 coloredlogs-15.0.1 confection-0.1.3 contourpy-1.2.0 cryptography-41.0.5 cycler-0.12.1 cymem-2.0.8 dataclasses-json-0.6.1 effdet-0.4.1 einops-0.7.0 emoji-2.8.0 et-xmlfile-1.1.0 exceptiongroup-1.1.3 faiss-cpu-1.7.4 fastapi-0.104.1 filetype-1.2.0 flatbuffers-23.5.26 fonttools-4.44.0 frozenlist-1.4.0 fschat-0.2.31 gitdb-4.0.11 gitpython-3.1.40 greenlet-3.0.1 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 huggingface-hub-0.17.3 humanfriendly-10.0 importlib-metadata-6.8.0 iniconfig-2.0.0 iopath-0.1.10 joblib-1.3.2 jsonpatch-1.33 jsonpointer-2.4 jsonschema-4.19.2 jsonschema-specifications-2023.7.1 kiwisolver-1.4.5 langchain-0.0.330 langchain-experimental-0.0.37 langcodes-3.3.0 langdetect-1.0.9 langsmith-0.0.57 layoutparser-0.3.4 lxml-4.9.3 markdown-3.5.1 markdown-it-py-3.0.0 markdown2-2.4.10 marshmallow-3.20.1 matplotlib-3.8.1 mdurl-0.1.2 msg-parser-1.2.0 multidict-6.0.4 murmurhash-1.0.10 mypy-extensions-1.0.0 nh3-0.2.14 nltk-3.8.1 numexpr-2.8.7 numpy-1.24.4 olefile-0.46 omegaconf-2.3.0 onnx-1.15.0 onnxruntime-1.15.1 openai-0.28.1 opencv-python-4.8.1.78 openpyxl-3.1.2 packaging-23.2 pandas-2.0.3 pathlib-1.0.1 pdf2image-1.16.3 pdfminer.six-20221105 pdfplumber-0.10.3 peft-0.6.0 pillow-9.5.0 pluggy-1.3.0 portalocker-2.8.2 preshed-3.0.9 prompt-toolkit-3.0.39 protobuf-4.25.0 psutil-5.9.6 pyarrow-14.0.0 pyclipper-1.3.0.post5 pycocotools-2.0.7 pycparser-2.21 pydantic-1.10.13 pydeck-0.8.1b0 pygments-2.16.1 pypandoc-1.12 pyparsing-3.1.1 pypdfium2-4.23.1 pyreadline3-3.4.1 pytesseract-0.3.10 pytest-7.4.3 python-dateutil-2.8.2 python-decouple-3.8 python-docx-1.1.0 python-iso639-2023.6.15 python-magic-0.4.27 python-magic-bin-0.4.14 python-multipart-0.0.6 python-pptx-0.6.21 pytz-2023.3.post1 pywin32-306 rapidfuzz-3.5.2 rapidocr_onnxruntime-1.3.8 referencing-0.30.2 regex-2023.10.3 rich-13.6.0 rpds-py-0.12.0 safetensors-0.4.0 scikit-learn-1.3.2 scipy-1.11.3 sentence_transformers-2.2.2 sentencepiece-0.1.99 shortuuid-1.0.11 simplejson-3.19.2 six-1.16.0 smart-open-6.4.0 smmap-5.0.1 sniffio-1.3.0 soupsieve-2.5 spacy-3.7.2 spacy-legacy-3.0.12 spacy-loggers-1.0.5 srsly-2.4.8 starlette-0.27.0 streamlit-1.28.1 streamlit-aggrid-0.3.4.post3 streamlit-antd-components-0.2.3 streamlit-chatbox-1.1.10 streamlit-option-menu-0.3.6 svgwrite-1.4.3 tabulate-0.9.0 tenacity-8.2.3 thinc-8.2.1 threadpoolctl-3.2.0 tiktoken-0.5.1 timm-0.9.10 tokenizers-0.14.1 toml-0.10.2 tomli-2.0.1 toolz-0.12.0 tornado-6.3.3 tqdm-4.66.1 transformers-4.35.0 transformers_stream_generator-0.0.4 typer-0.9.0 typing-inspect-0.9.0 tzdata-2023.3 tzlocal-5.2 unstructured-0.10.28 unstructured-inference-0.7.10 unstructured.pytesseract-0.3.12 uvicorn-0.23.2 validators-0.22.0 wasabi-1.1.2 watchdog-3.0.0 wavedrom-2.0.3.post3 wcwidth-0.2.9 weasel-0.3.3 websockets-12.0 xformers-0.0.22.post7 xlrd-2.0.1 yarl-1.9.2 zipp-3.17.0
(l2) D:\opt\l2\Langchain-Chatchat>
6、下载模型
下载两个模型：M3e-base内置模型和chatglm2-6b-int4模型。
https://huggingface.co/moka-ai/m3e-base
#备选
https://aistudio.baidu.com/datasetdetail/234251/0
#配置文件
https://huggingface.co/THUDM/chatglm2-6b-int4
7、修改配置文件
批量修改配置文件名
批量复制configs目录下所有的配置文件，去掉example后缀：
# cd Langchain-Chatchat
# 批量复制configs目录下所有配置文件，去掉example
python copy_config_example.py
(l2) D:\opt\l2\Langchain-Chatchat>python copy_config_example.py
(l2) D:\opt\l2\Langchain-Chatchat>
执行命令后：
修改model_config.py文件
修改m3e-base的模型本地路径：
MODEL_PATH = {
"embed_model": {
"ernie-tiny": "nghuyong/ernie-3.0-nano-zh",
"ernie-base": "nghuyong/ernie-3.0-base-zh",
"text2vec-base": "shibing624/text2vec-base-chinese",
"text2vec": "GanymedeNil/text2vec-large-chinese",
"text2vec-paraphrase": "shibing624/text2vec-base-chinese-paraphrase",
"text2vec-sentence": "shibing624/text2vec-base-chinese-sentence",
"text2vec-multilingual": "shibing624/text2vec-base-multilingual",
"text2vec-bge-large-chinese": "shibing624/text2vec-bge-large-chinese",
"m3e-small": "moka-ai/m3e-small",
# "m3e-base": "moka-ai/m3e-base",
# 修改为本地的 路径
"m3e-base": "E:\\llm_models\\m3e-base",
"m3e-large": "moka-ai/m3e-large",
"bge-small-zh": "BAAI/bge-small-zh",
"bge-base-zh": "BAAI/bge-base-zh",
"bge-large-zh": "BAAI/bge-large-zh",
"bge-large-zh-noinstruct": "BAAI/bge-large-zh-noinstruct",
"bge-base-zh-v1.5": "BAAI/bge-base-zh-v1.5",
"bge-large-zh-v1.5": "BAAI/bge-large-zh-v1.5",
"piccolo-base-zh": "sensenova/piccolo-base-zh",
"piccolo-large-zh": "sensenova/piccolo-large-zh",
"text-embedding-ada-002": "your OPENAI_API_KEY",
},
修改chatglm2-6b-int4模型本地路径：
"llm_model": {
# 以下部分模型并未完全测试，仅根据fastchat和vllm模型的模型列表推定支持
"chatglm-6b": "THUDM/chatglm-6b",
"chatglm2-6b": "THUDM/chatglm2-6b",
"chatglm2-6b-int4": "E:\\llm_models\\chatglm2-6b-int4", #"THUDM/chatglm2-6b-int4",
"chatglm2-6b-32k": "THUDM/chatglm2-6b-32k",
修改LLM模型为chatglm2-6b-int4：
# LLM 名称 ,修改为chatglm2-6b-int4
LLM_MODEL = "chatglm2-6b-int4"
修改LLM_DEVICE运行设备，看使用cpu、cuda（带GPU）或mps( mac本）
# 选用的 Embedding 名称
EMBEDDING_MODEL = "m3e-base" # 可以尝试最新的嵌入式sota模型：bge-large-zh-v1.5
# Embedding 模型运行设备。设为"auto"会自动检测，也可手动设定为"cuda","mps","cpu"其中之一。
EMBEDDING_DEVICE = "cpu"
# LLM 名称
LLM_MODEL = "chatglm2-6b-int4"
# LLM 运行设备。设为"auto"会自动检测，也可手动设定为"cuda","mps","cpu"其中之一。
LLM_DEVICE = "cpu"
修改server_config.py
0.2.6之前版本，需要修改0.0.0.0为127.0.0.1不然会报错
# 各服务器默认绑定host。如改为"0.0.0.0"需要修改下方所有XX_SERVER的host
DEFAULT_BIND_HOST = "127.0.0.1"
8、初始化数据库
(l2) D:\opt\l2\Langchain-Chatchat>python init_database.py --recreate-vs
database talbes reseted
recreating all vector stores
2023-11-06 18:21:06,059 - faiss_cache.py[line:75] - INFO: loading vector store in 'samples/vector_store' from disk.
2023-11-06 18:21:06,566 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: E:\llm_models\m3e-base
Batches: 100%|███████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 11.15it/s]
2023-11-06 18:21:11,388 - loader.py[line:54] - INFO: Loading faiss with AVX2 support.
2023-11-06 18:21:11,389 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to:
ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'")
2023-11-06 18:21:11,397 - loader.py[line:64] - INFO: Loading faiss.
2023-11-06 18:21:11,472 - loader.py[line:66] - INFO: Successfully loaded faiss.
2023-11-06 18:21:11,561 - faiss_cache.py[line:75] - INFO: loading vector store in 'samples/vector_store' from disk.
Batches: 100%|███████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.99it/s]
2023-11-06 18:21:11,713 - utils.py[line:287] - INFO: UnstructuredFileLoader used for D:\opt\l2\Langchain-Chatchat\knowledge_base\samples\content\test.txt
2023-11-06 18:21:17,676 - utils.py[line:160] - INFO: NumExpr defaulting to 8 threads.
2023-11-06 18:21:21,338 - font_manager.py[line:1578] - INFO: generated new fontManager
文档切分示例：page_content='ChatGPT是OpenAI开发的一个大型语言模型，可以提供各种主题的信息，\n# 如何向 ChatGPT 提问以获得高质量答案：提示技巧工程完全指南\n## 介绍\n我很高兴欢迎您阅读我的最新书籍《The Art of Asking ChatGPT for High-Quality Answers: A complete Guide to Prompt Engineering Techniques》。本书是一本全面指南，介绍了各种提示技术，用于从ChatGPT中生成 高质量的答案。\n我们将探讨如何使用不同的提示工程技术来实现不同的目标。ChatGPT是一款最先进的语言模型，能够生成类似人类的 文本。然而，理解如何正确地向ChatGPT提问以获得我们所需的高质量输出非常重要。而这正是本书的目的。' metadata={'source': 'D:\\opt\\l2\\Langchain-Chatchat\\knowledge_base\\samples\\content\\test.txt'}
正在将 samples/test.txt 添加到向量库，共包含66条文档
Batches: 100%|███████████████████████████████████████████████████████████████████████████| 3/3 [00:48<00:00, 16.10s/it]
2023-11-06 18:22:17,663 - faiss_cache.py[line:20] - INFO: 已将向量库 ('samples', 'vector_store') 保存到磁盘
总计用时： 0:01:11.719423
(l2) D:\opt\l2\Langchain-Chatchat>
9、一键启动项目
运行：
python startup.py -a
(l2) D:\opt\l2\Langchain-Chatchat>python startup.py -a
==============================Langchain-Chatchat Configuration==============================
操作系统：Windows-10-10.0.19044-SP0.
python版本：3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:34:57) [MSC v.1936 64 bit (AMD64)]
项目版本：v0.2.6
langchain版本：0.0.330. fastchat版本：0.2.31
当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['chatglm2-6b-int4'] @ cpu
{'device': 'cpu',
'host': '127.0.0.1',
'infer_turbo': False,
'model_path': 'E:\\llm_models\\chatglm2-6b-int4',
'port': 20002}
当前Embbedings模型： m3e-base @ cpu
==============================Langchain-Chatchat Configuration==============================
2023-11-06 18:35:19,714 - startup.py[line:626] - INFO: 正在启动服务：
2023-11-06 18:35:19,714 - startup.py[line:627] - INFO: 如需查看 llm_api 日志，请前往 D:\opt\l2\Langchain-Chatchat\logs
2023-11-06 18:35:27 | ERROR | stderr | [32mINFO[0m: Started server process [[36m16092[0m]
2023-11-06 18:35:27 | ERROR | stderr | [32mINFO[0m: Waiting for application startup.
2023-11-06 18:35:27 | ERROR | stderr | [32mINFO[0m: Application startup complete.
2023-11-06 18:35:27 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://127.0.0.1:20000[0m (Press CTRL+C to quit)
2023-11-06 18:35:35 | INFO | model_worker | Register to controller
2023-11-06 18:35:35 | INFO | model_worker | Loading the model ['chatglm2-6b-int4'] on worker f816f43b ...
2023-11-06 18:35:45 | INFO | model_worker | Register to controller
INFO: Started server process [18388]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:7861 (Press CTRL+C to quit)
==============================Langchain-Chatchat Configuration==============================
操作系统：Windows-10-10.0.19044-SP0.
python版本：3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:34:57) [MSC v.1936 64 bit (AMD64)]
项目版本：v0.2.6
langchain版本：0.0.330. fastchat版本：0.2.31
当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['chatglm2-6b-int4'] @ cpu
{'device': 'cpu',
'host': '127.0.0.1',
'infer_turbo': False,
'model_path': 'E:\\llm_models\\chatglm2-6b-int4',
'port': 20002}
当前Embbedings模型： m3e-base @ cpu
服务端运行信息：
OpenAI API Server: http://127.0.0.1:20000/v1
Chatchat API Server: http://127.0.0.1:7861
Chatchat WEBUI Server: http://127.0.0.1:8501
==============================Langchain-Chatchat Configuration==============================
You can now view your Streamlit app in your browser.
URL: http://127.0.0.1:8501
10、浏览器访问使用
11、安装成功
继续研究。
三、FAQ
1、Q：Failed to load cpm_kernels:No module named 'cpm_kernels'
2023-11-06 18:26:44 | INFO | model_worker | Loading the model ['chatglm2-6b-int4'] on worker ac9ad6b5 ...
2023-11-06 18:26:44 | INFO | model_worker | Register to controller
2023-11-06 18:26:49 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Failed to load cpm_kernels:No module named 'cpm_kernels'
A：pip3 install cpm_kernels -i https://pypi.tuna.tsinghua.edu.cn/simple/
(l2) D:\opt\l2\Langchain-Chatchat>pip3 install cpm_kernels -i https://pypi.tuna.tsinghua.edu.cn/simple/
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Collecting cpm_kernels
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/af/84/1831ce6ffa87b8fd4d9673c3595d0fc4e6631c0691eb43f406d3bf89b951/cpm_kernels-1.0.11-py3-none-any.whl (416 kB)
Installing collected packages: cpm_kernels
Successfully installed cpm_kernels-1.0.11