安装Paddle-ChatDocuments大模型

人工智能MOS2024-01-07 19:15

利用LangChain和ChatGLM-6B系列模型制作的Webui, 提供基于本地知识的大模型应用.

环境安装

项目依赖PaddlePaddle develolop版本和最新的PaddleNLP（更推荐在终端里安装）

安装PaddlePaddle Develop版本

In [1]

## 卸载环境中原有的旧PaddlePaddle版本
!pip uninstall paddlepaddle-gpu -y
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Found existing installation: paddlepaddle-gpu 0.0.0.post112
Uninstalling paddlepaddle-gpu-0.0.0.post112:
  Successfully uninstalled paddlepaddle-gpu-0.0.0.post112

In [2]

# 安装paddlepaddle-gpu develop版本
!python -m pip install paddlepaddle-gpu==0.0.0.post112 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html
Collecting paddlepaddle-gpu==0.0.0.post112
  Using cached https://paddle-wheel.bj.bcebos.com/develop/linux/linux-gpu-cuda11.2-cudnn8-mkl-gcc8.2-avx/paddlepaddle_gpu-0.0.0.post112-cp39-cp39-linux_x86_64.whl (679.5 MB)
Requirement already satisfied: paddle-bfloat==0.1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.1.7)
Requirement already satisfied: opt-einsum==3.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (3.3.0)
Requirement already satisfied: protobuf>=3.20.2 in ./.data/webide/pip/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (3.20.2)
Requirement already satisfied: requests>=2.20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (2.24.0)
Requirement already satisfied: numpy>=1.13 in ./.data/webide/pip/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (1.22.4)
Requirement already satisfied: astor in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.8.1)
Requirement already satisfied: decorator in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (4.4.2)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlepaddle-gpu==0.0.0.post112) (8.2.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in ./.data/webide/pip/lib/python3.9/site-packages (from requests>=2.20.0->paddlepaddle-gpu==0.0.0.post112) (2023.5.7)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Installing collected packages: paddlepaddle-gpu
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Successfully installed paddlepaddle-gpu-0.0.0.post112
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

安装PaddleNLP最新版本

In [ ]

## 安装方式1：源码安装
### clone PaddleNLP
%cd /home/aistudio
!git clone https://openi.pcl.ac.cn/PaddlePaddle/PaddleNLP.git -b develop

### 源码安装PaddleNLP
%cd /home/aistudio/PaddleNLP
!pip install -e . --user

In [3]

## 安装方式2：通过给定的.whl安装（更推荐）
%cd /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
!pip install ./paddlenlp-2.5.2.post0-py3-none-any.whl --user
复制代码 
/home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing ./paddlenlp-2.5.2.post0-py3-none-any.whl
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (4.1.0)
Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (2.4.0)
Requirement already satisfied: Flask-Babel in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (3.0.1)
Requirement already satisfied: protobuf==3.20.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (3.20.2)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (4.64.1)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.1.0)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.0.0)
Requirement already satisfied: rich in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (12.6.0)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (1.2.2)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.70.11.1)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.20.0)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (2.7.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.1.96)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.42.1)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.3.3)
Requirement already satisfied: huggingface-hub>=0.11.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.14.1)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.4.4)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.90.1)
Requirement already satisfied: typer in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from paddlenlp==2.5.2.post0) (0.7.0)
Requirement already satisfied: responses<0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (0.18.0)
Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (10.0.0)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (21.3)
Requirement already satisfied: numpy>=1.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.22.4)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.1.0)
Requirement already satisfied: pyyaml>=5.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (6.0)
Requirement already satisfied: requests>=2.19.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.24.0)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.8.3)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (2022.11.0)
Requirement already satisfied: pandas in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.5.3)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub>=0.11.1->paddlenlp==2.5.2.post0) (4.3.0)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub>=0.11.1->paddlenlp==2.5.2.post0) (3.9.0)
Requirement already satisfied: starlette<0.24.0,>=0.22.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->paddlenlp==2.5.2.post0) (0.23.1)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->paddlenlp==2.5.2.post0) (1.10.4)
Requirement already satisfied: Jinja2<4.0.0,>=3.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (3.1.2)
Requirement already satisfied: Babel<3.0.0,>=2.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2.11.0)
Requirement already satisfied: pytz<2023.0,>=2022.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2022.7.1)
Requirement already satisfied: Flask<3.0.0,>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask-Babel->paddlenlp==2.5.2.post0) (2.1.3)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich->paddlenlp==2.5.2.post0) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich->paddlenlp==2.5.2.post0) (2.13.0)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from seqeval->paddlenlp==2.5.2.post0) (0.24.2)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from typer->paddlenlp==2.5.2.post0) (8.0.4)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from uvicorn->paddlenlp==2.5.2.post0) (0.14.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (2.2.3)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (1.16.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (0.8.53)
Requirement already satisfied: Pillow>=7.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from visualdl->paddlenlp==2.5.2.post0) (8.2.0)
Requirement already satisfied: Werkzeug>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (2.1.2)
Requirement already satisfied: importlib-metadata>=3.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (6.0.0)
Requirement already satisfied: itsdangerous>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (2.1.2)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.3.0)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (22.1.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.7.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (4.0.2)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.1.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.5.2.post0) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from Jinja2<4.0.0,>=3.1.2->Flask-Babel->paddlenlp==2.5.2.post0) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from packaging->datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.0.9)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp==2.5.2.post0) (2023.5.7)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (1.2.0)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (1.6.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.5.2.post0) (2.1.0)
Requirement already satisfied: anyio<5,>=3.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from starlette<0.24.0,>=0.22.0->fastapi->paddlenlp==2.5.2.post0) (3.6.1)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from bce-python-sdk->visualdl->paddlenlp==2.5.2.post0) (3.9.9)
Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from bce-python-sdk->visualdl->paddlenlp==2.5.2.post0) (0.18.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib->visualdl->paddlenlp==2.5.2.post0) (0.10.0)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from anyio<5,>=3.4.0->starlette<0.24.0,>=0.22.0->fastapi->paddlenlp==2.5.2.post0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from importlib-metadata>=3.6.0->Flask<3.0.0,>=2.0.0->Flask-Babel->paddlenlp==2.5.2.post0) (3.8.1)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from kiwisolver>=1.0.1->matplotlib->visualdl->paddlenlp==2.5.2.post0) (56.2.0)
paddlenlp is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

【重要】安装完环境后请重启一下notebook内核

In [1]

# 测试ChatGLM-6B的Paddle版本是否工作
from paddlenlp import Taskflow
chatbot = Taskflow("text2text_generation",batch_size=2)
results = chatbot(['您好！','很高兴认识你'])
print(results)
复制代码 
[2023-06-02 17:23:28,324] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/ice_text.model and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,414] [    INFO] - Downloading ice_text.model from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/ice_text.model
复制代码 
  0%|          | 0.00/2.58M [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:28,838] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/added_tokens.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,896] [ WARNING] - file<https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/added_tokens.json> not exist
[2023-06-02 17:23:28,899] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/special_tokens_map.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:28,962] [ WARNING] - file<https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/special_tokens_map.json> not exist
[2023-06-02 17:23:28,965] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/tokenizer_config.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1
[2023-06-02 17:23:29,026] [    INFO] - Downloading tokenizer_config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/tokenizer_config.json
复制代码 
  0%|          | 0.00/441 [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:29,540] [    INFO] - Downloading config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/config.json
复制代码 
  0%|          | 0.00/773 [00:00<?, ?B/s]
复制代码 
[2023-06-02 17:23:29,707] [    INFO] - loading configuration file /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b-v1.1/config.json
[2023-06-02 17:23:29,710] [    INFO] - Model config ChatGLMConfig {
  "_name_or_path": "THUDM/chatglm-6b",
  "activation": "gelu",
  "architectures": [
    "ChatGLMModel"
  ],
  "attention_scale": true,
  "auto_map": {
    "AutoConfig": "configuration_chatglm.ChatGLMConfig",
    "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration",
    "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration"
  },
  "bos_token_id": 130004,
  "dtype": "float16",
  "eos_token_id": 130005,
  "gmask_token_id": 130001,
  "hidden_size": 4096,
  "inner_hidden_size": 16384,
  "layernorm_epsilon": 1e-05,
  "mask_token_id": 130000,
  "max_sequence_length": 2048,
  "model_type": "chatglm",
  "num_attention_heads": 32,
  "num_hidden_layers": 28,
  "num_image_tokens": 0,
  "output_predict": true,
  "pad_token_id": 3,
  "paddlenlp_version": null,
  "position_encoding_2d": true,
  "pre_seq_len": null,
  "prefix_projection": false,
  "quantization_bit": 0,
  "recompute": false,
  "transformers_version": "4.23.1",
  "use_cache": true,
  "vocab_size": 130528
}

[2023-06-02 17:23:29,771] [    INFO] - Downloading model_state.pdparams from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b-v1.1/model_state.pdparams
复制代码 
  0%|          | 0.00/12.5G [00:00<?, ?B/s]
复制代码 
W0602 17:28:29.657855  3296 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0602 17:28:29.661559  3296 gpu_resources.cc:149] device: 0, cuDNN Version: 8.2.
[2023-06-02 17:28:41,939] [ WARNING] - Some weights of the model checkpoint at THUDM/chatglm-6b-v1.1 were not used when initializing ChatGLMForConditionalGeneration: ['transformer.layers.24.attention.rotary_emb.inv_freq', 'transformer.layers.25.attention.rotary_emb.inv_freq', 'transformer.layers.0.attention.rotary_emb.inv_freq', 'transformer.layers.21.attention.rotary_emb.inv_freq', 'transformer.layers.19.attention.rotary_emb.inv_freq', 'transformer.layers.8.attention.rotary_emb.inv_freq', 'transformer.layers.11.attention.rotary_emb.inv_freq', 'transformer.layers.27.attention.rotary_emb.inv_freq', 'transformer.layers.22.attention.rotary_emb.inv_freq', 'transformer.layers.20.attention.rotary_emb.inv_freq', 'transformer.layers.1.attention.rotary_emb.inv_freq', 'transformer.layers.3.attention.rotary_emb.inv_freq', 'transformer.layers.15.attention.rotary_emb.inv_freq', 'transformer.layers.26.attention.rotary_emb.inv_freq', 'transformer.layers.9.attention.rotary_emb.inv_freq', 'transformer.
- This IS expected if you are initializing ChatGLMForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ChatGLMForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[2023-06-02 17:28:41,942] [ WARNING] - Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at THUDM/chatglm-6b-v1.1 and are newly initialized: ['transformer.layers.21.attention.rotary_embeddings.inv_freq', 'transformer.layers.27.attention.rotary_embeddings.inv_freq', 'transformer.layers.13.attention.rotary_embeddings.inv_freq', 'transformer.layers.10.attention.rotary_embeddings.inv_freq', 'transformer.layers.9.attention.rotary_embeddings.inv_freq', 'transformer.layers.22.attention.rotary_embeddings.inv_freq', 'transformer.layers.18.attention.rotary_embeddings.inv_freq', 'transformer.layers.2.attention.rotary_embeddings.inv_freq', 'transformer.layers.23.attention.rotary_embeddings.inv_freq', 'transformer.layers.20.attention.rotary_embeddings.inv_freq', 'transformer.layers.16.attention.rotary_embeddings.inv_freq', 'transformer.layers.17.attention.rotary_embeddings.inv_freq', 'transformer.layers.0.attention.rotary_embeddings.inv_freq', 'transform
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
复制代码 
{'result': ['您好!有什么我能为您效劳的吗?', '你好!我也很高兴认识你。作为一名人工智能助手,我很高兴能够与你交流和回答你的问题。请问有什么我可以为你效劳的吗?']}

In [2]

# 安装依赖（时间较长）
%cd /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
!pip install -r requirements.txt
复制代码 
/home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: langchain==0.0.147 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (0.0.147)
Requirement already satisfied: unstructured[local-inference] in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (0.6.5)
Requirement already satisfied: layoutparser[layoutmodels,tesseract] in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (0.3.4)
Requirement already satisfied: nltk in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (3.4.5)
Requirement already satisfied: beautifulsoup4 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (4.11.1)
Requirement already satisfied: icetk in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (0.0.4)
Requirement already satisfied: cpm_kernels in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (1.0.11)
Collecting faiss-cpu
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b5/0c/97faf412f20b3e0f52c02c894871f3213c72c73d57d1f039fe867d3c3437/faiss_cpu-1.7.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/17.6 MB ? eta -:--:--━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.9/17.6 MB 25.5 MB/s eta 0:00:01━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/17.6 MB 17.9 MB/s eta 0:00:01━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/17.6 MB 26.7 MB/s eta 0:00:01━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/17.6 MB 22.9 MB/s eta 0:00:01━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/17.6 MB 23.2 MB/s eta 0:00:01━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/17.6 MB 22.4 MB/s eta 0:00:01━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/17.6 MB 19.4 MB/s eta 0:00:01━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/17.6 MB 22.5 MB/s eta 0:00:01━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 7.2/17.6 MB 23.0 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 8.8/17.6 MB 25.4 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━ 9.1/17.6 MB 25.3 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 10.7/17.6 MB 26.0 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━
Requirement already satisfied: protobuf in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (3.20.2)
Requirement already satisfied: openapi-schema-pydantic<2.0,>=1.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.2.4)
Requirement already satisfied: requests<3,>=2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: PyYAML>=5.4.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (6.0)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (3.8.3)
Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (4.0.2)
Requirement already satisfied: SQLAlchemy<2,>=1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.4.41)
Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (0.5.7)
Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (8.2.2)
Requirement already satisfied: tqdm>=4.48.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (4.64.1)
Requirement already satisfied: numpy<2,>=1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.22.4)
Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (2.8.4)
Requirement already satisfied: pydantic<2,>=1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from langchain==0.0.147->-r requirements.txt (line 1)) (1.10.4)
Requirement already satisfied: python-pptx in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.6.21)
Requirement already satisfied: python-magic in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.4.27)
Requirement already satisfied: markdown in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (3.1.1)
Requirement already satisfied: pandas in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.5.3)
Requirement already satisfied: pdfminer.six in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (20221105)
Collecting python-docx
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/8b/a0/52729ce4aa026f31b74cc877be1d11e4ddeaa361dc7aebec148171644b33/python-docx-0.8.11.tar.gz (5.6 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/5.6 MB ? eta -:--:--━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.8/5.6 MB 27.1 MB/s eta 0:00:01━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/5.6 MB 25.6 MB/s eta 0:00:01━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/5.6 MB 16.8 MB/s eta 0:00:01━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/5.6 MB 16.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━ 3.2/5.6 MB 18.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━ 3.5/5.6 MB 17.1 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 4.5/5.6 MB 19.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 4.5/5.6 MB 19.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 18.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi>=2022.12.07 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (2023.5.7)
Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (3.0.5)
Requirement already satisfied: lxml in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (4.9.1)
Requirement already satisfied: pypandoc in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.11)
Requirement already satisfied: argilla in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.7.0)
Requirement already satisfied: msg-parser in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (1.2.0)
Requirement already satisfied: pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (8.2.0)
Requirement already satisfied: unstructured-inference==0.4.4 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured[local-inference]->-r requirements.txt (line 2)) (0.4.4)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.90.1)
Requirement already satisfied: onnxruntime in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (1.14.1)
Requirement already satisfied: huggingface-hub in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.14.1)
Requirement already satisfied: python-multipart in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.0.5)
Requirement already satisfied: opencv-python!=4.7.0.68 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (4.6.0.66)
Requirement already satisfied: transformers in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (4.29.1)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.20.0)
Requirement already satisfied: pdf2image in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.16.3)
Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.6.3)
Requirement already satisfied: iopath in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.1.10)
Requirement already satisfied: pdfplumber in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.9.0)
Requirement already satisfied: torchvision in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.15.2)
Requirement already satisfied: effdet in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.3.0)
Requirement already satisfied: torch in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.1)
Requirement already satisfied: pytesseract in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.3.10)
Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nltk->-r requirements.txt (line 4)) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from beautifulsoup4->-r requirements.txt (line 5)) (2.3.2.post1)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from icetk->-r requirements.txt (line 6)) (3.9.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from icetk->-r requirements.txt (line 6)) (0.1.96)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (22.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.7.2)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.2.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (6.0.2)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.147->-r requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: marshmallow-enum<2.0.0,>=1.5.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (1.5.1)
Requirement already satisfied: typing-inspect>=0.4.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (0.8.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.3.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (3.19.0)
Requirement already satisfied: typing-extensions>=4.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pydantic<2,>=1->langchain==0.0.147->-r requirements.txt (line 1)) (4.3.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (1.25.11)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (2.8)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from requests<3,>=2->langchain==0.0.147->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from SQLAlchemy<2,>=1->langchain==0.0.147->-r requirements.txt (line 1)) (2.0.2)
Requirement already satisfied: wrapt<1.15,>=1.13 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.14.1)
Requirement already satisfied: typer<1.0.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.7.0)
Requirement already satisfied: monotonic in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.6)
Requirement already satisfied: httpx<0.24,>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.23.3)
Requirement already satisfied: backoff in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (2.2.1)
Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (21.3)
Requirement already satisfied: rich<=13.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (12.6.0)
Requirement already satisfied: deprecated~=1.2.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.2.13)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pandas->unstructured[local-inference]->-r requirements.txt (line 2)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pandas->unstructured[local-inference]->-r requirements.txt (line 2)) (2022.7.1)
Requirement already satisfied: omegaconf>=2.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.3.0)
Requirement already satisfied: timm>=0.4.12 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.6.13)
Requirement already satisfied: pycocotools>=2.0.2 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.6)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.99)
Requirement already satisfied: sympy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.11.1)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.10.3.66)
Requirement already satisfied: networkx in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.4)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.14.3)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.101)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (10.2.10.91)
Requirement already satisfied: triton==2.0.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.0)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.4.91)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.7.91)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (8.5.0.96)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (10.9.0.58)
Requirement already satisfied: jinja2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (11.4.0.1)
Requirement already satisfied: wheel in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.36.2)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (56.2.0)
Requirement already satisfied: cmake in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from triton==2.0.0->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (3.26.3)
Requirement already satisfied: lit in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from triton==2.0.0->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (16.0.3)
Requirement already satisfied: portalocker in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from iopath->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.7.0)
Requirement already satisfied: olefile>=0.46 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from msg-parser->unstructured[local-inference]->-r requirements.txt (line 2)) (0.46)
Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from openpyxl->unstructured[local-inference]->-r requirements.txt (line 2)) (1.4.1)
Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from openpyxl->unstructured[local-inference]->-r requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: cryptography>=36.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (38.0.1)
Collecting Wand>=0.6.10
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fa/a8/f76bbafa1a42f1ed45ece3c2b2e4f70a5d6cdb5f38bb0780db8efcdc58ab/Wand-0.6.11-py2.py3-none-any.whl (143 kB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/143.6 kB ? eta -:--:--━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.6/143.6 kB 14.2 MB/s eta 0:00:00
Collecting pillow
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ff/fc/48a51c0fe2a00d5def57b9981a1e0f8339b516351da7a51500383d833bc8/Pillow-9.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
     l     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/3.3 MB ? eta -:--:--━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/3.3 MB 33.6 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━ 2.4/3.3 MB 34.7 MB/s eta 0:00:01━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 32.6 MB/s eta 0:00:00
Requirement already satisfied: XlsxWriter>=0.5.7 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from python-pptx->unstructured[local-inference]->-r requirements.txt (line 2)) (3.1.0)
Requirement already satisfied: cffi>=1.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from cryptography>=36.0.0->pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (1.15.1)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.5.0)
Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.16.3)
Requirement already satisfied: sniffio in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (1.3.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from omegaconf>=2.0->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (4.9.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from packaging>=20.0->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (3.0.9)
Requirement already satisfied: matplotlib>=2.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.2.3)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich<=13.0.1->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (2.13.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from rich<=13.0.1->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (0.9.1)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from typer<1.0.0,>=0.6.0->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (8.0.4)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from typing-inspect>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.147->-r requirements.txt (line 1)) (1.0.0)
Requirement already satisfied: starlette<0.24.0,>=0.22.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from fastapi->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.23.1)
Requirement already satisfied: fsspec in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from huggingface-hub->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (2022.11.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from jinja2->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (2.0.1)
Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from networkx->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (4.4.2)
Requirement already satisfied: flatbuffers in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (23.5.9)
Requirement already satisfied: coloredlogs in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (15.0.1)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from sympy->torch->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.2.1)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from transformers->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.13.3)
Requirement already satisfied: regex!=2019.12.17 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from transformers->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (2023.5.5)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from uvicorn->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (0.14.0)
Requirement already satisfied: pycparser in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six->unstructured[local-inference]->-r requirements.txt (line 2)) (2.21)
Requirement already satisfied: anyio<5.0,>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from httpcore<0.17.0,>=0.15.0->httpx<0.24,>=0.15->argilla->unstructured[local-inference]->-r requirements.txt (line 2)) (3.6.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (from matplotlib>=2.1.0->pycocotools>=2.0.2->effdet->layoutparser[layoutmodels,tesseract]->-r requirements.txt (line 3)) (0.10.0)
Requirement already satisfied: humanfriendly>=9.1 in /home/aistudio/.data/webide/pip/lib/python3.9/site-packages (from coloredlogs->onnxruntime->unstructured-inference==0.4.4->unstructured[local-inference]->-r requirements.txt (line 2)) (10.0)
Building wheels for collected packages: python-docx
  Building wheel for python-docx (setup.py) ... done
  Created wheel for python-docx: filename=python_docx-0.8.11-py3-none-any.whl size=184492 sha256=aebcd1a7557e583667bd320214d70c6839872eb717a2946608c92223921de6ba
  Stored in directory: /home/aistudio/.cache/pip/wheels/39/ca/c1/d3e7abe5ce2e8423382d816e689c056bc26590f48fad8f20ac
Successfully built python-docx
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Installing collected packages: Wand, faiss-cpu, python-docx, pillow
  Attempting uninstall: pillow
    WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
    Found existing installation: Pillow 8.2.0
    Uninstalling Pillow-8.2.0:
      Successfully uninstalled Pillow-8.2.0
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
streamlit 1.13.0 requires protobuf!=3.20.2,<4,>=3.12, but you have protobuf 3.20.2 which is incompatible.
parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 23.2.1 which is incompatible.
Successfully installed Wand-0.6.11 faiss-cpu-1.7.4 pillow-9.5.0 python-docx-0.8.11
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

In [3]

!pip uninstall numpy -y
!pip install numpy
复制代码 
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Found existing installation: numpy 1.22.4
Uninstalling numpy-1.22.4:
  Successfully uninstalled numpy-1.22.4
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages (1.23.5)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)
WARNING: Ignoring invalid distribution -umpy (/opt/conda/envs/python35-paddle120-env/lib/python3.9/site-packages)

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip

In [ ]

# 执行命令行脚本
!python demo.py

In [ ]

# 安装方式1：通过源码安装paddlenlp-pipelines
%cd /home/aistudio/PaddleNLP/pipelines
!pip install -e . --user

In [ ]

# 安装方式2：通过.whl安装paddlenlp-pipelines
%cd /home/aistudio/
!pip install /home/aistudio/LangChain-ChatGLM-Webui/paddlepaddle/paddle_pipelines-0.5.2-py3-none-any.whl --user

In [ ]

# 运行非LangChain版本的本地知识库自动问答
%cd /home/aistudio/
!python chat_documents.py

Github地址

  1. LangChain-ChatGLM-Webui: 基于LangChain和ChatGLM等LLM实现本地知识库的自动问答；
  2. Paddle-ChatDocuments：不使用LangChain，完全使用PaddlePaddle生态实现基于本地知识库的自动问答。

引用

  1. ChatGLM-6B: ChatGLM-6B: 开源双语对话语言模型

  2. LangChain: Building applications with LLMs through composability

    @inproceedings{
    zeng2023glm-130b,
    title={{GLM}-130B: An Open Bilingual Pre-trained Model},
    author={Aohan Zeng and Xiao Liu and Zhengxiao Du and Zihan Wang and Hanyu Lai and Ming Ding and Zhuoyi Yang and Yifan Xu and Wendi Zheng and Xiao Xia and Weng Lam Tam and Zixuan Ma and Yufei Xue and Jidong Zhai and Wenguang Chen and Zhiyuan Liu and Peng Zhang and Yuxiao Dong and Jie Tang},
    booktitle={The Eleventh International Conference on Learning Representations (ICLR)},
    year={2023},
    url={https://openreview.net/forum?id=-Aw0rrrPUF}
    }

    @inproceedings{du2022glm,
    title={GLM: General Language Model Pretraining with Autoregressive Blank Infilling},
    author={Du, Zhengxiao and Qian, Yujie and Liu, Xiao and Ding, Ming and Qiu, Jiezhong and Yang, Zhilin and Tang, Jie},
    booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
    pages={320--335},
    year={2022}
    }

相关推荐
jk_1019 分钟前
MATLAB中bicgstabl函数用法
开发语言·算法·matlab
白葵新10 分钟前
Open3D mesh 计算点云模型的表面积及体积
人工智能·python·算法·计算机视觉·3d
茗创科技13 分钟前
拓扑感知多模态融合在神经动态表征学习和分类中的应用
人工智能·学习·分类
清上尘17 分钟前
数学建模之数据分析【九】：数据清理总结
人工智能·机器学习·数学建模·数据挖掘·数据分析
jmoych18 分钟前
PPT分享：某集团公司供应链-销售与运营计划S&OP
大数据·人工智能
2401_8576226621 分钟前
Ruby跨平台移动应用开发的新篇章
开发语言·后端·ruby
国中之林28 分钟前
【C++ Primer Plus习题】4.10
开发语言·c++·学习·刷题
Aaron_94529 分钟前
Cargo: Rust的包管理和构建工具
开发语言·rust·log4j
算力资源比较多31 分钟前
国内外大模型汇总：Open AI大模型、Google大模型、Microsoft大模型、文心一言大模型、通义千问大模型、字节豆包大模型、智普清言大模型
人工智能·语言模型·文心一言·循环神经网络
小蜗的房子36 分钟前
C语言01 每日一练01
c语言·开发语言·数据结构·学习·算法·编程题目
热门推荐
01第14届蓝桥杯青少组python试题解析：23年5月省赛02RAG 实践- Ollama+RagFlow 部署本地知识库032024电赛H题参考方案（+视频演示）——自动行使小车04Coze扣子平台完整体验和实践（附国内和国际版对比）05组基轨迹建模 GBTM的介绍与实现（Stata 或 R）06带你看懂思源字体的各个版本都有什么区别07YzmCMS 7.0任意函数调用RCE 漏洞研究分析08yolov8实战第五天——yolov8+ffmpg实时视频流检测并进行实时推流——（推流，保姆教学）09解决Pycharm下载torch报错：OSError: [WinError 126] 找不到指定的模块。10【极空间NAS】使用WebDAV服务 + DDNSTO内网穿透 实现思源笔记的内外网同步