前言
Docker 是目前最流行的容器平台，而 NVIDIA Container Toolkit 可以让 Docker 容器直接访问 GPU，实现 AI、深度学习和高性能计算工作负载。
在企业内网环境中，通常无法直接联网下载 RPM 包或使用官方仓库，所以需要提前准备好：
Docker RPM 包及依赖
NVIDIA Container Toolkit RPM 包及依赖
NVIDIA 驱动（确保 GPU 能正常使用）一、离线安装docker
1.下载 Docker
下载 Docker 的最新版本 docker-20.10.9.tgz 到本地。
https://download.docker.com/linux/static/stable/x86_64/
https://github.com/docker-archive/docker-ce/releases
2. 安装 Docker
解压下载的 Docker 压缩包
tar xvf docker-20.10.9.tgz
复制 Docker 目录到 /usr/bin，使 Docker 命令可执行
cp -r docker/* /usr/bin
清理解压后的文件
rm -rf docker docker-20.10.9.tgz
3. 配置 Docker 服务
创建 Docker 服务配置文件
sudo vim /etc/systemd/system/docker.service
在文件中添加以下内容：
bash
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target赋予 Docker 服务文件执行权限
sudo chmod +x /etc/systemd/system/docker.service
重新加载 systemd 配置
sudo systemctl daemon-reload
启用 Docker 服务开机自启
sudo systemctl enable docker.service
启动 Docker 服务
sudo systemctl start docker
检查 Docker 服务状态
sudo systemctl status docker
4. 添加执行权限
赋予 Docker 服务文件执行权限
sudo chmod +x /etc/systemd/system/docker.service
重新加载 systemd 配置
sudo systemctl daemon-reload
5. 启动docker
启用 Docker 服务开机自启
sudo systemctl enable docker.service
启动 Docker 服务
sudo systemctl start docker
检查 Docker 服务状态
sudo systemctl status docker
二、离线安装NVIDIA Container Toolkit
通过安装和配置 NVIDIA Container Toolkit，可以使 Docker 容器使用宿主机的 GPU 资源。，NVIDIA Container Toolkit 是 nvidia-docker 的升级版，建议使用NVIDIA Container Toolkit而不是旧的nvidia-docker。1.下载 NVIDIA Container Toolkit
下载 Docker 的最新版本 nvidia-container-toolkit_1.18.1_rpm_x86_64.tar.gz 到本地。
https://github.com/NVIDIA/nvidia-container-toolkit/releases
2. 安装 NVIDIA Container Toolkit
bash
[root@localhost source]# tar -zxvf nvidia-container-toolkit_1.18.1_rpm_x86_64.tar.gz
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container-devel-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container-libseccomp2-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container-static-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container-tools-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container1-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/libnvidia-container1-debuginfo-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/nvidia-container-toolkit-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/nvidia-container-toolkit-base-1.18.1-1.x86_64.rpm
release-v1.18.1-stable/packages/centos7/x86_64/nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64.rpm
[root@localhost source]#
[root@localhost llama3.1-8b]# dnf localinstall -y release-v1.18.1-stable/packages/centos7/x86_64/*.rpm
Last metadata expiration check: 0:34:34 ago on Wed 31 Dec 2025 03:54:57 PM CST.
Dependencies resolved.
=======================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
=======================================================================================================================================================================================================================================================================================
Installing:
libnvidia-container-devel x86_64 1.18.1-1 @commandline 27 k
libnvidia-container-static x86_64 1.18.1-1 @commandline 157 k
libnvidia-container-tools x86_64 1.18.1-1 @commandline 47 k
libnvidia-container1 x86_64 1.18.1-1 @commandline 1.1 M
libnvidia-container1-debuginfo x86_64 1.18.1-1 @commandline 387 k
nvidia-container-toolkit x86_64 1.18.1-1 @commandline 1.6 M
nvidia-container-toolkit-base x86_64 1.18.1-1 @commandline 8.5 M
nvidia-container-toolkit-operator-extensions x86_64 1.18.1-1 @commandline 4.4 M
Transaction Summary
=======================================================================================================================================================================================================================================================================================
Install 8 Packages
Total size: 16 M
Installed size: 47 M
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Running scriptlet: libnvidia-container1-1.18.1-1.x86_64 1/1
Running scriptlet: nvidia-container-toolkit-base-1.18.1-1.x86_64 1/1
Running scriptlet: libnvidia-container-devel-1.18.1-1.x86_64 1/1
Running scriptlet: libnvidia-container-tools-1.18.1-1.x86_64 1/1
Running scriptlet: nvidia-container-toolkit-1.18.1-1.x86_64 1/1
Running scriptlet: libnvidia-container-static-1.18.1-1.x86_64 1/1
Running scriptlet: nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 1/1
Running scriptlet: libnvidia-container1-debuginfo-1.18.1-1.x86_64 1/1
Preparing : 1/1
Running scriptlet: libnvidia-container1-1.18.1-1.x86_64 1/8
Installing : libnvidia-container1-1.18.1-1.x86_64 1/8
Running scriptlet: libnvidia-container1-1.18.1-1.x86_64 1/8
Running scriptlet: nvidia-container-toolkit-base-1.18.1-1.x86_64 2/8
Installing : nvidia-container-toolkit-base-1.18.1-1.x86_64 2/8
Running scriptlet: nvidia-container-toolkit-base-1.18.1-1.x86_64 2/8
Created symlink /etc/systemd/system/multi-user.target.wants/nvidia-cdi-refresh.path → /etc/systemd/system/nvidia-cdi-refresh.path.
Created symlink /etc/systemd/system/multi-user.target.wants/nvidia-cdi-refresh.service → /etc/systemd/system/nvidia-cdi-refresh.service.
Regenerating NVIDIA CDI specification...
Running scriptlet: libnvidia-container-devel-1.18.1-1.x86_64 3/8
Installing : libnvidia-container-devel-1.18.1-1.x86_64 3/8
Running scriptlet: libnvidia-container-devel-1.18.1-1.x86_64 3/8
Running scriptlet: libnvidia-container-tools-1.18.1-1.x86_64 4/8
Installing : libnvidia-container-tools-1.18.1-1.x86_64 4/8
Running scriptlet: libnvidia-container-tools-1.18.1-1.x86_64 4/8
Running scriptlet: nvidia-container-toolkit-1.18.1-1.x86_64 5/8
Installing : nvidia-container-toolkit-1.18.1-1.x86_64 5/8
Running scriptlet: nvidia-container-toolkit-1.18.1-1.x86_64 5/8
Running scriptlet: libnvidia-container-static-1.18.1-1.x86_64 6/8
Installing : libnvidia-container-static-1.18.1-1.x86_64 6/8
Running scriptlet: libnvidia-container-static-1.18.1-1.x86_64 6/8
Running scriptlet: nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 7/8
Installing : nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 7/8
Running scriptlet: nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 7/8
Running scriptlet: libnvidia-container1-debuginfo-1.18.1-1.x86_64 8/8
Installing : libnvidia-container1-debuginfo-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container1-debuginfo-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container1-1.18.1-1.x86_64 8/8
Running scriptlet: nvidia-container-toolkit-base-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container-devel-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container-tools-1.18.1-1.x86_64 8/8
Running scriptlet: nvidia-container-toolkit-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container-static-1.18.1-1.x86_64 8/8
Running scriptlet: nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 8/8
Running scriptlet: libnvidia-container1-debuginfo-1.18.1-1.x86_64 8/8
Verifying : libnvidia-container1-1.18.1-1.x86_64 1/8
Verifying : libnvidia-container1-debuginfo-1.18.1-1.x86_64 2/8
Verifying : libnvidia-container-devel-1.18.1-1.x86_64 3/8
Verifying : libnvidia-container-static-1.18.1-1.x86_64 4/8
Verifying : libnvidia-container-tools-1.18.1-1.x86_64 5/8
Verifying : nvidia-container-toolkit-1.18.1-1.x86_64 6/8
Verifying : nvidia-container-toolkit-base-1.18.1-1.x86_64 7/8
Verifying : nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64 8/8
Installed:
libnvidia-container-devel-1.18.1-1.x86_64 libnvidia-container-static-1.18.1-1.x86_64 libnvidia-container-tools-1.18.1-1.x86_64 libnvidia-container1-1.18.1-1.x86_64 libnvidia-container1-debuginfo-1.18.1-1.x86_64 nvidia-container-toolkit-1.18.1-1.x86_64
nvidia-container-toolkit-base-1.18.1-1.x86_64 nvidia-container-toolkit-operator-extensions-1.18.1-1.x86_64
Complete!
[root@localhost llama3.1-8b]#3.配置 Docker 使用 NVIDIA runtime
编辑 /etc/docker/daemon.json 文件：
bash
{
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}或者使用指令配置
sudo nvidia-ctk runtime configure --runtime=docker --set-as-default
重启docker生效
sudo systemctl restart docker
bash
[root@localhost llama3.1-8b]# nvidia-ctk runtime configure --runtime=docker --set-as-default
INFO[0000] Config file does not exist; using empty config
INFO[0000] Wrote updated config to /etc/docker/daemon.json
INFO[0000] It is recommended that docker daemon be restarted.
[root@localhost llama3.1-8b]#
[root@localhost llama3.1-8b]# systemctl restart docker
[root@localhost llama3.1-8b]#