centos-LLM-生物信息-BioGPT安装

参考 :

GitHub - microsoft/BioGPT

https://github.com/microsoft/BioGPT

BioGPT:用于生物医学文本生成和挖掘的生成式预训练转换器 |生物信息学简报 |牛津学术 --- BioGPT: generative pre-trained transformer for biomedical text generation and mining | Briefings in Bioinformatics | Oxford Academic

https://academic.oup.com/bib/article/23/6/bbac409/6713511

No such file or directory: 'fairseq\version.txt · Issue #5518 · facebookresearch/fairseq

https://github.com/facebookresearch/fairseq/issues/5518

环境 :

centos 7,anaconda3,CUDA 11.6


目录

问题描述

BioGPT,是一个在大规模生物医学文献上预先训练的特定领域的生成 Transformer 语言模型。

本文介绍了它在本地服务器上的安装过程,下一篇文章将会写具体用法。

安装

要求:(来自github页面readme)

  1. Python version == 3.10
  2. PyTorch version == 1.12.0
  3. fairseq version == 0.12.0(要先装PyTorch)
  4. Moses
  5. fastBPE
  6. sacremoses
  7. sklearn

PyTorch

安装 CUDA Toolkit 11.6

手动下载 CUDA Toolkit 11.6

shell 复制代码
wget https://developer.download.nvidia.com/compute/cuda/11.6.0/local_installers/cuda_11.6.0_510.39.01_linux.run
sudo sh cuda_11.6.0_510.39.01_linux.run

安装的时候会有选项,这样选:

然后选择Install。

这个要选择哪个?选择Choose components to upgrade

shell 复制代码
Existing installation of CUDA Toolkit 11.6 found:                            
x Upgrade all                                                                  
x Choose components to upgrade                                                 
x No, abort installation   

设置环境变量:

shell 复制代码
echo 'export PATH=/usr/local/cuda-11.6/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

安装PyTorch

使用清华源。

shell 复制代码
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0

检查PyTorch能否识别GPU:输出True就对了

shell 复制代码
python -c "import torch; print(torch.cuda.is_available())"

fairseq

shell 复制代码
pip --default-timeout=1000 install fairseq -i https://pypi.tuna.tsinghua.edu.cn/simple

有可能需要:

shell 复制代码
sudo yum install -y python3-devel gcc g++ make cmake
pip install cython -i https://pypi.tuna.tsinghua.edu.cn/simple

官方的做法是下面。但我试了,1是要科学上网,2是会报错:TimeoutError: The read operation timed out或者ERROR: Could not find a version that satisfies the requirement cython (from versions: none)。安装了cython也不行。

shell 复制代码
git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./

# on MacOS:
# CFLAGS="-stdlib=libc++" pip install --editable ./

# to install the latest stable release (0.10.x)
# pip install fairseq

报错:FileNotFoundError: Errno 2 No such file or directory: 'fairseq/version.txt'

完整报错:

shell 复制代码
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Traceback (most recent call last):
        File "/home/xxx/anaconda3/envs/biogpt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/home/xxx/anaconda3/envs/biogpt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
        File "/home/xxx/anaconda3/envs/biogpt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-ubaynszi/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-ubaynszi/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-ubaynszi/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 27, in <module>
        File "<string>", line 18, in write_version_py
      FileNotFoundError: [Errno 2] No such file or directory: 'fairseq/version.txt'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

原因:参考下面的链接,是pip版本太高了,我的pip是25.0.1,降到24就可以了。

No such file or directory: 'fairseq\version.txt · Issue #5518 · facebookresearch/fairseq

https://github.com/facebookresearch/fairseq/issues/5518

shell 复制代码
python -m pip install pip==24.0 

Moses

shell 复制代码
git clone https://github.com/moses-smt/mosesdecoder.git
export MOSES=${PWD}/mosesdecoder

fastBPE

shell 复制代码
git clone https://github.com/glample/fastBPE.git
export FASTBPE=${PWD}/fastBPE
cd fastBPE
g++ -std=c++11 -pthread -O3 fastBPE/main.cc -IfastBPE -o fast

sacremoses

shell 复制代码
pip install sacremoses

sklearn

shell 复制代码
pip install scikit-learn -i https://pypi.tuna.tsinghua.edu.cn/simple
相关推荐
戊本菌17 分钟前
进程(三)进程优先级:谁先上 CPU
linux·c语言·硬件架构
Forget the Dream41 分钟前
Spring AI 2.0 系列(二):第一个 ChatClient——从注入到一次完整调用
java·人工智能·spring boot·ai·springai
aningx1 小时前
Arch 系用 greetd 实现开机自动进 Plasma Wayland,注销自动回桌面
linux
郝学胜-神的一滴1 小时前
Effective Python 条款35:用类封装替代生成器异常注入
开发语言·人工智能·python·金融·pandas·scikit-learn·matplotlib
火山引擎开发者社区1 小时前
豆包工作,3 个月,800 万,20 亿
人工智能
Chuck.li2 小时前
AI 长篇创作工具选型:工作流对照与自测清单
人工智能·大模型·ai写作·网文作者·ai网文·豆包写作
aleimu2 小时前
Windows 11 右键打开 Cmder 2.0:从找不到菜单到默认使用 Cmder
前端·人工智能·后端
江无行者2 小时前
Linux 服务器 top 命令详解
linux
SwanArc技术笔记2 小时前
读技术文档,先用哪个 AI?模型指南筛选与 GPT-6.2 Astra 实操
人工智能
深海鱼在掘金2 小时前
深入浅出RAG——第21章:回顾与展望
人工智能