统信UOS V2500服务器操作系统+海光K100 AI卡环境下VLLM服务部署

部署环境说明

服务器： 曙光 X7850H0（256 核+ 8张A100+1.5T内存）
服务器操作系统： UOS V2500
DTK版本： 25.04.1
Rocm版本： 6.3.13
VLLM版本： 0.8.5
Python版本： 3.10.16
注：海光vllm安装文件依赖numa库与python3.11版本不兼容，因此python选择3.10版本

安装包清单

驱动包： rock-6.3.13-V1.12.0.run
开发工具包： DTK-25.04.1-openEuler22.03-x86_64.tar.gz
Python 依赖库：

amdsmi-24.5.3+02cbffb.dirty-py3-none-any.whl

dropout_layer_norm-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

flash_attn-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

flash_mla-1.0.0+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

fused_dense_lib-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

grouped_gemm-0.5.0+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl

grouped_gemm_int4-0.5.0+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl

lmslim-0.3.0+das.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

moe_w8a8-0.0.1+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl

moe_w8a8_prefill_gemm-0.0.1+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl

rotary_emb-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

torch-2.4.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

triton-3.0.0+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

vllm-0.8.5.post1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl

xentropy_cuda_lib-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl
Python 源码包： Python-3.10.16.tar.xz
注：上述文件除Python源码包外，其他包都需要自行到海光开源社区下载。

物理机环境部署

安装系统依赖包

复制代码

yum install -y gcc
yum install -y openssl-devel
yum install -y zlib-devel
yum install -y bzip2-devel
yum install -y gdbm-devel
yum install -y xz-devel
yum install -y sqlite-devel
yum install -y tk-devel
yum install -y libuuid-devel
yum install -y libnsl2-devel
yum install -y readline-devel
yum install -y libffi-devel
yum install -y numactl-devel.x86_64
yum install -y pciutils
yum install -y libgfortran.x86_64
yum install -y automake
yum install -y findutils
yum install -y mlocate
yum install -y autoconf
yum install -y rpm-build
yum install -y gcc-c++ 
yum install -y libdrm-devel.x86_64

Python3.10 源码安装

复制代码

# 解压缩Python3.10压缩包
tar -xvf Python-3.10.16.tar.xz 

# 进入解压缩后python目录
cd Python-3.10.16

# 配置Python编译参数：
# --prefix=/usr/local/python3.10  指定安装目录为/usr/local/python3.10
# --with-ensurepip=install  确保安装pip工具（Python包管理工具）
# --enable-shared  编译生成共享库（.so文件），供其他程序调用Python库
./configure --prefix=/usr/local/python3.10 --with-ensurepip=install --enable-shared

# 编译源码：
# -j$(nproc)  表示使用当前系统所有可用CPU核心进行并行编译，加速编译过程
# $(nproc)会自动获取CPU核心数，例如8核CPU会展开为-j8
make -j$(nproc)

# 安装编译好的Python到指定目录（即--prefix指定的/usr/local/python3.10）
# 会将可执行文件、库文件、头文件等复制到目标路径
make install

# 设置Python环境变量 
echo "export LD_LIBRARY_PATH=/usr/local/python3.10/lib:\$LD_LIBRARY_PATH" >> /etc/profile 
echo "export PATH=/usr/local/python3.10/bin:\$PATH" >> /etc/profile 
echo "export LD_LIBRARY_PATH=/usr/local/python3.10/lib:\$LD_LIBRARY_PATH" >> /etc/bashrc 
echo "export PATH=/usr/local/python3.10/bin:\$PATH" >> /etc/bashrc

# 验证python安装是否成功
python3 --version
pip3 --version

Rcom 驱动安装

复制代码

# 修改安装文件权限
chmod +x rock-6.3.13-V1.12.0.run

# 安装驱动
./rock-6.3.13-V1.12.0.run

# 重启主机
reboot

DTK 安装

复制代码

# 解压缩文件到/opt目录
tar -zxf /opt/dcu/DTK-25.04.1-openEuler22.03-x86_64.tar.gz -C /opt

# 设置环境参数
echo "export LIBRARY_PATH=/opt/dtk-25.04.1/lib:\$LIBRARY_PATH" >> /etc/profile
echo "export LIBRARY_PATH=/opt/dtk-25.04.1/lib:\$LIBRARY_PATH" >> /etc/bashrc 
echo "source /opt/dtk-25.04.1/env.sh" >> /etc/bashrc

vllm服务安装

复制代码

pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple torch-2.4.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple dropout_layer_norm-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple fused_dense_lib-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple grouped_gemm-0.5.0+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple grouped_gemm_int4-0.5.0+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple moe_w8a8-0.0.1+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple moe_w8a8_prefill_gemm-0.0.1+das.dtk2504-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple rotary_emb-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple triton-3.0.0+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple xentropy_cuda_lib-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple flash_mla-1.0.0+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple lmslim-0.3.0+das.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple flash_attn-2.6.1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple amdsmi-24.5.3+02cbffb.dirty-py3-none-any.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple vllm-0.8.5.post1+das.opt1.dtk25041-cp310-cp310-manylinux_2_28_x86_64.whl 
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple ray[default]==2.51.0

VLLM服务验证与运行

复制代码

# 验证vllm部署是否成功
vllm --version
# vll部署Deek服务
vllm serve /opt/models/DeepSeek-R1-Distill-Qwen-32B --trust-remote-code  --dtype float16 --max-model-len 1024  --tensor-parallel-size 8 --host 0.0.0.0 --port 8000 --served-model-name deepseek