安装Paddle-ChatDocuments大模型

人工智能MOS2024-01-07 19:15

利用LangChain和ChatGLM-6B系列模型制作的Webui, 提供基于本地知识的大模型应用.

环境安装

项目依赖PaddlePaddle develolop版本和最新的PaddleNLP（更推荐在终端里安装）

安装PaddlePaddle Develop版本

In [1]

## 卸载环境中原有的旧PaddlePaddle版本
!pip uninstall paddlepaddle-gpu -y
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Found existing installation: paddlepaddle-gpu 0.0.0.post112
Uninstalling paddlepaddle-gpu-0.0.0.post112:
  Successfully uninstalled paddlepaddle-gpu-0.0.0.post112

In [2]

# 安装paddlepaddle-gpu develop版本
!python -m pip install paddlepaddle-gpu==0.0.0.post112 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html
Collecting paddlepaddle-gpu==0.0.0.post112
  Using cached https://paddle-wheel.bj.bcebos.com/develop/linux/linux-gpu-cuda11.2-cudnn8-mkl-gcc8.2-avx/paddlepaddle_gpu-0.0.0.post112-cp39-cp39-linux_x86_64.whl (679.5 MB)
Requirement already satisfied: paddle-bfloat==0.1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.1.7)
Requirement already satisfied: opt-einsum==3.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (3.3.0)
Requirement already satisfied: protobuf>=3.20.2 in ./.data/webide/pip/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (3.20.2)
Requirement already satisfied: requests>=2.20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (2.24.0)
Requirement already satisfied: numpy>=1.13 in ./.data/webide/pip/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (1.22.4)
Requirement already satisfied: astor in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.8.1)
Requirement already satisfied: decorator in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (4.4.2)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (8.2.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in ./.data/webide/pip/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (2023.5.7)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Installing collected packages: paddlepaddle-gpu
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Successfully installed paddlepaddle-gpu-0.0.0.post112
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

安装PaddleNLP最新版本

In [ ]

## 安装方式1：源码安装
### clone PaddleNLP
%cd /home/aistudio
!git clone https://openi.pcl.ac.cn/PaddlePaddle/PaddleNLP.git -b develop

### 源码安装PaddleNLP
%cd /home/aistudio/PaddleNLP
!pip install -e . --user

In [3]

## 安装方式2：通过给定的.whl安装（更推荐）
%cd /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
!pip install ./paddlenlp-2.5.2.post0-py3-none-any.whl --user
复制代码 
/home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing ./paddlenlp-2.5.2.post0-py3-none-any.whl
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (4.1.0)
Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (2.4.0)
Requirement already satisfied: Flask-Babel in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (3.0.1)
Requirement already satisfied: protobuf==3.20.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (3.20.2)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (4.64.1)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.1.0)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.0.0)
Requirement already satisfied: rich in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (12.6.0)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.2.2)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.70.11.1)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.20.0)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (2.7.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.1.96)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.42.1)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.3.3)
Requirement already satisfied: huggingface-hub>=0.11.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.14.1)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.4.4)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.90.1)
Requirement already satisfied: typer in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.7.0)
Requirement already satisfied: responses<0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (0.18.0)
Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (10.0.0)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (21.3)
Requirement already satisfied: numpy>=1.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.22.4)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.1.0)
Requirement already satisfied: pyyaml>=5.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (6.0)
Requirement already satisfied: requests>=2.19.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.24.0)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.8.3)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (2022.11.0)
Requirement already satisfied: pandas in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.5.3)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub>=0.11.1->paddlenlp==2.5.2.post0) (4.3.0)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub>=0.11.1->paddlenlp==2.5.2.post0) (3.9.0)
Requirement already satisfied: starlette<0.24.0,>=0.22.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->paddlenlp==2.5.2.post0) (0.23.1)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->paddlenlp==2.5.2.post0) (1.10.4)
Requirement already satisfied: Jinja2<4.0.0,>=3.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (3.1.2)
Requirement already satisfied: Babel<3.0.0,>=2.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2.11.0)
Requirement already satisfied: pytz<2023.0,>=2022.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2022.7.1)
Requirement already satisfied: Flask<3.0.0,>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2.1.3)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich->paddlenlp==2.5.2.post0) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich->paddlenlp==2.5.2.post0) (2.13.0)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from seqeval->paddlenlp==2.5.2.post0) (0.24.2)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from typer->paddlenlp==2.5.2.post0) (8.0.4)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from uvicorn->paddlenlp==2.5.2.post0) (0.14.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (2.2.3)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (1.16.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (0.8.53)
Requirement already satisfied: Pillow>=7.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (8.2.0)
Requirement already satisfied: Werkzeug>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (2.1.2)
Requirement already satisfied: importlib-metadata>=3.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (6.0.0)
Requirement already satisfied: itsdangerous>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (2.1.2)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.3.0)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (22.1.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.7.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (4.0.2)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.1.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Jinja2<4.0.0,>=3.1.2->Flask-Babel->paddlenlp==2.5.2.post0) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from packaging->datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.0.9)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2023.5.7)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (1.2.0)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (1.6.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (2.1.0)
Requirement already satisfied: anyio<5,>=3.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from starlette<0.24.0,>=0.22.0->fastapi->paddlenlp==2.5.2.post0) (3.6.1)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from bce-python-sdk->visualdl->paddlenlp==2.5.2.post0) (3.9.9)
Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from bce-python-sdk->visualdl->paddlenlp==2.5.2.post0) (0.18.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (0.10.0)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from anyio<5,>=3.4.0->starlette<0.24.0,>=0.22.0->fastapi->paddlenlp==2.5.2.post0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from importlib-metadata>=3.6.0->Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (3.8.1)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from kiwisolver>=1.0.1->matplotlib->visualdl->paddlenlp==2.5.2.post0) (56.2.0)
paddlenlp is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

【重要】安装完环境后请重启一下notebook内核

In [1]

# 测试ChatGLM-6B的Paddle版本是否工作
from paddlenlp import Taskflow
chatbot = Taskflow("text2text_generation",batch_size=2)
results = chatbot(['您好！','很高兴认识你'])
print(results)
复制代码 
[2023-06-02 17:23:28,324] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/ice_text.model and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,414] [    INFO] - Downloading ice_text.model from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/ice_text.model
复制代码 
  0%|          | 0.00/2.58M [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:28,838] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/added_tokens.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,896] [ WARNING] - file<https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/added_tokens.json> not exist
[2023-06-02 17:23:28,899] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/special_tokens_map.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,962] [ WARNING] - file<https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/special_tokens_map.json> not exist
[2023-06-02 17:23:28,965] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/tokenizer_config.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:29,026] [    INFO] - Downloading tokenizer_config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/tokenizer_config.json
复制代码 
  0%|          | 0.00/441 [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:29,540] [    INFO] - Downloading config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/config.json
复制代码 
  0%|          | 0.00/773 [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:29,707] [    INFO] - loading configuration file /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1/config.json
[2023-06-02 17:23:29,710] [    INFO] - Model config ChatGLMConfig {
  "_name_or_path": "THUDM/chatglm-6b",
  "activation": "gelu",
  "architectures": [
    "ChatGLMModel"
  ],
  "attention_scale": true,
  "auto_map": {
    "AutoConfig": "configuration_chatglm.ChatGLMConfig",
    "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration",
    "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration"
  },
  "bos_token_id": 130004,
  "dtype": "float16",
  "eos_token_id": 130005,
  "gmask_token_id": 130001,
  "hidden_size": 4096,
  "inner_hidden_size": 16384,
  "layernorm_epsilon": 1e-05,
  "mask_token_id": 130000,
  "max_sequence_length": 2048,
  "model_type": "chatglm",
  "num_attention_heads": 32,
  "num_hidden_layers": 28,
  "num_image_tokens": 0,
  "output_predict": true,
  "pad_token_id": 3,
  "paddlenlp_version": null,
  "position_encoding_2d": true,
  "pre_seq_len": null,
  "prefix_projection": false,
  "quantization_bit": 0,
  "recompute": false,
  "transformers_version": "4.23.1",
  "use_cache": true,
  "vocab_size": 130528
}

[2023-06-02 17:23:29,771] [    INFO] - Downloading model_state.pdparams from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/model_state.pdparams
复制代码 
  0%|          | 0.00/12.5G [00:00<?, ?B/s]
复制代码 
W0602 17:28:29.657855  3296 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0602 17:28:29.661559  3296 gpu_resources.cc:149] device: 0, cuDNN Version: 8.2.
[2023-06-02 17:28:41,939] [ WARNING] - Some weights of the model checkpoint at THUDM/chatglm-6b-v1.1 were not used when initializing ChatGLMForConditionalGeneration: ['transformer.layers.24.attention.rotary_emb.inv_freq', 'transformer.layers.25.attention.rotary_emb.inv_freq', 'transformer.layers.0.attention.rotary_emb.inv_freq', 'transformer.layers.21.attention.rotary_emb.inv_freq', 'transformer.layers.19.attention.rotary_emb.inv_freq', 'transformer.layers.8.attention.rotary_emb.inv_freq', 'transformer.layers.11.attention.rotary_emb.inv_freq', 'transformer.layers.27.attention.rotary_emb.inv_freq', 'transformer.layers.22.attention.rotary_emb.inv_freq', 'transformer.layers.20.attention.rotary_emb.inv_freq', 'transformer.layers.1.attention.rotary_emb.inv_freq', 'transformer.layers.3.attention.rotary_emb.inv_freq', 'transformer.layers.15.attention.rotary_emb.inv_freq', 'transformer.layers.26.attention.rotary_emb.inv_freq', 'transformer.layers.9.attention.rotary_emb.inv_freq', 'transformer.
- This IS expected if you are initializing ChatGLMForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ChatGLMForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[2023-06-02 17:28:41,942] [ WARNING] - Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at THUDM/chatglm-6b-v1.1 and are newly initialized: ['transformer.layers.21.attention.rotary_embeddings.inv_freq', 'transformer.layers.27.attention.rotary_embeddings.inv_freq', 'transformer.layers.13.attention.rotary_embeddings.inv_freq', 'transformer.layers.10.attention.rotary_embeddings.inv_freq', 'transformer.layers.9.attention.rotary_embeddings.inv_freq', 'transformer.layers.22.attention.rotary_embeddings.inv_freq', 'transformer.layers.18.attention.rotary_embeddings.inv_freq', 'transformer.layers.2.attention.rotary_embeddings.inv_freq', 'transformer.layers.23.attention.rotary_embeddings.inv_freq', 'transformer.layers.20.attention.rotary_embeddings.inv_freq', 'transformer.layers.16.attention.rotary_embeddings.inv_freq', 'transformer.layers.17.attention.rotary_embeddings.inv_freq', 'transformer.layers.0.attention.rotary_embeddings.inv_freq', 'transform
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
复制代码 
{'result': ['您好!有什么我能为您效劳的吗?', '你好!我也很高兴认识你。作为一名人工智能助手,我很高兴能够与你交流和回答你的问题。请问有什么我可以为你效劳的吗?']}

In [2]

# 安装依赖（时间较长）
%cd /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
!pip install -r requirements.txt
复制代码 
/home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: langchain==0.0.147 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (0.0.147)
Requirement already satisfied: unstructured[local-inference] in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (0.6.5)
Requirement already satisfied: layoutparser[layoutmodels,tesseract] in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (0.3.4)
Requirement already satisfied: nltk in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (3.4.5)
Requirement already satisfied: beautifulsoup4 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (4.11.1)
Requirement already satisfied: icetk in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (0.0.4)
Requirement already satisfied: cpm_kernels in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (1.0.11)
Collecting faiss-cpu
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b5/0c/97faf412f20b3e0f52c02c894871f3213c72c73d57d1f039fe867d3c3437/faiss_cpu-1.7.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/17.6 MB ? eta -:--:--━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.9/17.6 MB 25.5 MB/s eta 0:00:01━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/17.6 MB 17.9 MB/s eta 0:00:01━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/17.6 MB 26.7 MB/s eta 0:00:01━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/17.6 MB 22.9 MB/s eta 0:00:01━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/17.6 MB 23.2 MB/s eta 0:00:01━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/17.6 MB 22.4 MB/s eta 0:00:01━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/17.6 MB 19.4 MB/s eta 0:00:01━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/17.6 MB 22.5 MB/s eta 0:00:01━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 7.2/17.6 MB 23.0 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 8.8/17.6 MB 25.4 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━ 9.1/17.6 MB 25.3 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 10.7/17.6 MB 26.0 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━
Requirement already satisfied: protobuf in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (3.20.2)
Requirement already satisfied: openapi-schema-pydantic<2.0,>=1.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.2.4)
Requirement already satisfied: requests<3,>=2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: PyYAML>=5.4.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (6.0)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (3.8.3)
Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (4.0.2)
Requirement already satisfied: SQLAlchemy<2,>=1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.4.41)
Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (0.5.7)
Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (8.2.2)
Requirement already satisfied: tqdm>=4.48.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (4.64.1)
Requirement already satisfied: numpy<2,>=1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.22.4)
Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (2.8.4)
Requirement already satisfied: pydantic<2,>=1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.10.4)
Requirement already satisfied: python-pptx in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.6.21)
Requirement already satisfied: python-magic in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.4.27)
Requirement already satisfied: markdown in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (3.1.1)
Requirement already satisfied: pandas in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.5.3)
Requirement already satisfied: pdfminer.six in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (20221105)
Collecting python-docx
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/8b/a0/52729ce4aa026f31b74cc877be1d11e4ddeaa361dc7aebec148171644b33/python-docx-0.8.11.tar.gz (5.6 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/5.6 MB ? eta -:--:--━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.8/5.6 MB 27.1 MB/s eta 0:00:01━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/5.6 MB 25.6 MB/s eta 0:00:01━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/5.6 MB 16.8 MB/s eta 0:00:01━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/5.6 MB 16.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━ 3.2/5.6 MB 18.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━ 3.5/5.6 MB 17.1 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 4.5/5.6 MB 19.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 4.5/5.6 MB 19.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 18.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi>=2022.12.07 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (2023.5.7)
Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (3.0.5)
Requirement already satisfied: lxml in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (4.9.1)
Requirement already satisfied: pypandoc in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.11)
Requirement already satisfied: argilla in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.7.0)
Requirement already satisfied: msg-parser in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.2.0)
Requirement already satisfied: pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (8.2.0)
Requirement already satisfied: unstructured-inference==0.4.4 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.4.4)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.90.1)
Requirement already satisfied: onnxruntime in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (1.14.1)
Requirement already satisfied: huggingface-hub in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.14.1)
Requirement already satisfied: python-multipart in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.0.5)
Requirement already satisfied: opencv-python!=4.7.0.68 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (4.6.0.66)
Requirement already satisfied: transformers in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (4.29.1)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.20.0)
Requirement already satisfied: pdf2image in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.16.3)
Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.6.3)
Requirement already satisfied: iopath in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.1.10)
Requirement already satisfied: pdfplumber in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.9.0)
Requirement already satisfied: torchvision in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.15.2)
Requirement already satisfied: effdet in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.3.0)
Requirement already satisfied: torch in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.1)
Requirement already satisfied: pytesseract in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.3.10)
Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nltk->-r requirements.txt (line 4)) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from beautifulsoup4->-r requirements.txt (line 5)) (2.3.2.post1)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from icetk->-r requirements.txt (line 6)) (3.9.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from icetk->-r requirements.txt (line 6)) (0.1.96)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (22.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.7.2)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.2.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (6.0.2)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: marshmallow-enum<2.0.0,>=1.5.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (1.5.1)
Requirement already satisfied: typing-inspect>=0.4.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (0.8.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.3.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (3.19.0)
Requirement already satisfied: typing-extensions>=4.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pydantic<2,>=1->langchain==0.0.147->-r requirements.txt (line 1)) (4.3.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (1.25.11)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (2.8)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from SQLAlchemy<2,>=1->langchain==0.0.147->-r requirements.txt (line 1)) (2.0.2)
Requirement already satisfied: wrapt<1.15,>=1.13 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.14.1)
Requirement already satisfied: typer<1.0.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.7.0)
Requirement already satisfied: monotonic in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.6)
Requirement already satisfied: httpx<0.24,>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.23.3)
Requirement already satisfied: backoff in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (2.2.1)
Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (21.3)
Requirement already satisfied: rich<=13.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (12.6.0)
Requirement already satisfied: deprecated~=1.2.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.2.13)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pandas->unstructured[local-inference]->-r requirements.txt (line 2)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pandas->unstructured[local-inference]->-r requirements.txt (line 2)) (2022.7.1)
Requirement already satisfied: omegaconf>=2.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.3.0)
Requirement already satisfied: timm>=0.4.12 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.6.13)
Requirement already satisfied: pycocotools>=2.0.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.6)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.99)
Requirement already satisfied: sympy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.11.1)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.10.3.66)
Requirement already satisfied: networkx in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.4)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.14.3)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.101)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (10.2.10.91)
Requirement already satisfied: triton==2.0.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.0)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.4.91)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.91)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (8.5.0.96)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (10.9.0.58)
Requirement already satisfied: jinja2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.4.0.1)
Requirement already satisfied: wheel in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.36.2)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (56.2.0)
Requirement already satisfied: cmake in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from triton==2.0.0->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (3.26.3)
Requirement already satisfied: lit in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from triton==2.0.0->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (16.0.3)
Requirement already satisfied: portalocker in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from iopath->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.7.0)
Requirement already satisfied: olefile>=0.46 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from msg-parser->unstructured[local-inference]->-r requirements.txt (line 2)) (0.46)
Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from openpyxl->unstructured[local-inference]->-r requirements.txt (line 2)) (1.4.1)
Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from openpyxl->unstructured[local-inference]->-r requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: cryptography>=36.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (38.0.1)
Collecting Wand>=0.6.10
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fa/a8/f76bbafa1a42f1ed45ece3c2b2e4f70a5d6cdb5f38bb0780db8efcdc58ab/Wand-0.6.11-py2.py3-none-any.whl (143 kB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/143.6 kB ? eta -:--:--━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.6/143.6 kB 14.2 MB/s eta 0:00:00
Collecting pillow
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ff/fc/48a51c0fe2a00d5def57b9981a1e0f8339b516351da7a51500383d833bc8/Pillow-9.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/3.3 MB ? eta -:--:--━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/3.3 MB 33.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━ 2.4/3.3 MB 34.7 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 32.6 MB/s eta 0:00:00
Requirement already satisfied: XlsxWriter>=0.5.7 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from python-pptx->unstructured[local-inference]->-r requirements.txt (line 2)) (3.1.0)
Requirement already satisfied: cffi>=1.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from cryptography>=36.0.0->pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (1.15.1)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.5.0)
Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.16.3)
Requirement already satisfied: sniffio in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.3.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from omegaconf>=2.0->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (4.9.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from packaging>=20.0->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (3.0.9)
Requirement already satisfied: matplotlib>=2.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.2.3)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich<=13.0.1->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (2.13.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich<=13.0.1->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.9.1)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from typer<1.0.0,>=0.6.0->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (8.0.4)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from typing-inspect>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (1.0.0)
Requirement already satisfied: starlette<0.24.0,>=0.22.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.23.1)
Requirement already satisfied: fsspec in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (2022.11.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from jinja2->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.1)
Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from networkx->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (4.4.2)
Requirement already satisfied: flatbuffers in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (23.5.9)
Requirement already satisfied: coloredlogs in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (15.0.1)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from sympy->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.2.1)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from transformers->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.13.3)
Requirement already satisfied: regex!=2019.12.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from transformers->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (2023.5.5)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from uvicorn->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.14.0)
Requirement already satisfied: pycparser in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (2.21)
Requirement already satisfied: anyio<5.0,>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpcore<0.17.0,>=0.15.0->httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (3.6.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.10.0)
Requirement already satisfied: humanfriendly>=9.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from coloredlogs->onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (10.0)
Building wheels for collected packages: python-docx
  Building wheel for python-docx (setup.py) ... done
  Created wheel for python-docx: filename=python_docx-0.8.11-py3-none-any.whl size=184492 sha256=aebcd1a7557e583667bd320214d70c6839872eb717a2946608c92223921de6ba
  Stored in directory: /home/aistudio/.cache/pip/wheels/39/ca/c1/d3e7abe5ce2e8423382d816e689c056bc26590f48fad8f20ac
Successfully built python-docx
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Installing collected packages: Wand, faiss-cpu, python-docx, pillow
  Attempting uninstall: pillow
    WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
    Found existing installation: Pillow 8.2.0
    Uninstalling Pillow-8.2.0:
      Successfully uninstalled Pillow-8.2.0
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
streamlit 1.13.0 requires protobuf!=3.20.2,<4,>=3.12, but you have protobuf 3.20.2 which is incompatible.
parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 23.2.1 which is incompatible.
Successfully installed Wand-0.6.11 faiss-cpu-1.7.4 pillow-9.5.0 python-docx-0.8.11
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

In [3]

!pip uninstall numpy -y
!pip install numpy
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Found existing installation: numpy 1.22.4
Uninstalling numpy-1.22.4:
  Successfully uninstalled numpy-1.22.4
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (1.23.5)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

In [ ]

# 执行命令行脚本
!python demo.py

In [ ]

# 安装方式1：通过源码安装paddlenlp-pipelines
%cd /home/aistudio/PaddleNLP/pipelines
!pip install -e . --user

In [ ]

# 安装方式2：通过.whl安装paddlenlp-pipelines
%cd /home/aistudio/
!pip install /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle/paddle_pipelines-0.5.2-py3-none-any.whl --user

In [ ]

# 运行非LangChain版本的本地知识库自动问答
%cd /home/aistudio/
!python chat_documents.py

Github地址

  1. LangChain-ChatGLM-Webui: 基于LangChain和ChatGLM等LLM实现本地知识库的自动问答；
  2. Paddle-ChatDocuments：不使用LangChain，完全使用PaddlePaddle生态实现基于本地知识库的自动问答。

引用

  1. ChatGLM-6B: ChatGLM-6B: 开源双语对话语言模型

  2. LangChain: Building applications with LLMs through composability

    @inproceedings{
    zeng2023glm-130b,
    title={{GLM}-130B: An Open Bilingual Pre-trained Model},
    author={Aohan Zeng and Xiao Liu and Zhengxiao Du and Zihan Wang and Hanyu Lai and Ming Ding and Zhuoyi Yang and Yifan Xu and Wendi Zheng and Xiao Xia and Weng Lam Tam and Zixuan Ma and Yufei Xue and Jidong Zhai and Wenguang Chen and Zhiyuan Liu and Peng Zhang and Yuxiao Dong and Jie Tang},
    booktitle={The Eleventh International Conference on Learning Representations (ICLR)},
    year={2023},
    url={https://openreview.net/forum?id=-Aw0rrrPUF}
    }

    @inproceedings{du2022glm,
    title={GLM: General Language Model Pretraining with Autoregressive Blank Infilling},
    author={Du, Zhengxiao and Qian, Yujie and Liu, Xiao and Ding, Ming and Qiu, Jiezhong and Yang, Zhilin and Tang, Jie},
    booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
    pages={320--335},
    year={2022}
    }

相关推荐
Buleall3 分钟前
期末考学C
java·开发语言
重生之绝世牛码5 分钟前
Java设计模式 —— 【结构型模式】外观模式详解
java·大数据·开发语言·设计模式·设计原则·外观模式
小蜗牛慢慢爬行11 分钟前
有关异步场景的 10 大 Spring Boot 面试问题
java·开发语言·网络·spring boot·后端·spring·面试
Algorithm157620 分钟前
云原生相关的 Go 语言工程师技术路线(含博客网址导航）
开发语言·云原生·golang
边缘计算社区24 分钟前
首个！艾灵参编的工业边缘计算国家标准正式发布
大数据·人工智能·边缘计算
shinelord明30 分钟前
【再谈设计模式】享元模式~对象共享的优化妙手
开发语言·数据结构·算法·设计模式·软件工程
游客52035 分钟前
opencv中的各种滤波器简介
图像处理·人工智能·python·opencv·计算机视觉
一位小说男主35 分钟前
编码器与解码器：从‘乱码’到‘通话’
人工智能·深度学习
Monly2136 分钟前
Java(若依)：修改Tomcat的版本
java·开发语言·tomcat
热门推荐
01〔AI 绘画〕Stable Diffusion 之 解决绘制多人或面部很小的人物时面部崩坏问题 篇02ARM学习（31）编译器对overlay方式的支持03玄机平台应急响应—webshell查杀04DisplayPort AltMode On Type-c协议_Ver2.0(学习笔记)05校验 GPT-4 真实性的三个经典问题：快速区分 GPT-3.5 与 GPT-4，并提供免费测试网站06Docker 夺命连环 15 问076个主流的工业3D管道设计软件08基于大语言模型（LLM）的合成数据生成、策展和评估的综述09(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明10红米手机使用google play