背景
某个机房的几台机器前段时间通过dnat暴露至公网后被入侵挖矿，为避免一些安全隐患将这几台机器执行重装系统操作；
这里主要记录配置nvidia驱动及cuda&anaconda。
步骤
大概分为几个步骤
- 禁用nouveau
- 配置grub显示菜单
- install nvidia-driver
- install cuda
- install anaconda
- 测试
执行
system info
bash
root@xxx:~# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
root@xxx:~# uname -a
Linux exai-121 6.5.0-41-generic #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux禁用nouveau
bash
echo -e "blacklist nouveau\noptions nouveau modeset=0" > /etc/modprobe.d/blacklist-nouveau.conf
update-initramfs -u配置grub显示菜单
bash
sed -i 's/^GRUB_TIMEOUT=.*$/GRUB_TIMEOUT=10/' /etc/default/grub
sed -i 's/^GRUB_TIMEOUT_STYLE=hidden/#&/' /etc/default/grub
# 验证
grep -E "GRUB_TIMEOUT" /etc/default/grub
# 更新grub
update-grubinstall nvidia-driver
两种安装驱动的方法
- 添加ppa源的方式用apt安装
- 官网下载指定版本的离线安全包安装
方法一
使用ubuntu带的硬件扫描配合ppa源安装
bash
# 扫描硬件
ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:01.1/0000:01:00.0 ==
modalias : pci:v000010DEd00002684sv000010DEsd0000167Cbc03sc00i00
vendor : NVIDIA Corporation
manual_install: True
driver : nvidia-driver-535-server-open - distro non-free
driver : nvidia-driver-535 - distro non-free recommended
driver : nvidia-driver-545-open - distro non-free
driver : nvidia-driver-535-server - distro non-free
driver : nvidia-driver-545 - distro non-free
driver : nvidia-driver-535-open - distro non-free
driver : nvidia-driver-530 - third-party non-free
driver : xserver-xorg-video-nouveau - distro free builtin
# apt安装需要的驱动版本即可 例如
apt install nvidia-driver-535 -y方法二
编译器下载
bash
apt update && apt install gcc make -y官网下载535.183.01版本对应的离线安装run包
这里直接使用curl+proxy下载了
bash
chmod +x NVIDIA-Linux-x86_64-535.183.01.run
./NVIDIA-Linux-x86_64-535.183.01.run过程中遇到的选项根据提示选择即可
bash
root@xxx:~# nvidia-smi
Thu Jul 14 09:xx:01 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:01:00.0 Off | Off |
| 32% 62C P2 224W / 450W | 19736MiB / 24564MiB | 96% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 Off | 00000000:81:00.0 Off | Off |
| 31% 49C P2 144W / 450W | 11430MiB / 24564MiB | 38% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce RTX 4090 Off | 00000000:C1:00.0 Off | Off |
| 37% 63C P2 140W / 450W | 11430MiB / 24564MiB | 39% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 3 NVIDIA GeForce RTX 4090 Off | 00000000:C2:00.0 Off | Off |
| 32% 58C P2 136W / 450W | 11430MiB / 24564MiB | 40% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+其他常用命令
更详细的卸载请参考https://blog.csdn.net/qq_43652666/article/details/134705794
bash
# 卸载nvidia驱动
nvidia-uninstall如果系统没有nvidia-uninstall可以使用dpkg包管理器列出系统中的nvidia关键字的包进行卸载
bash
dpkg -l |grep nvidia*
apt remove --purge '^nvidia-.*'install cuda
在https://developer.nvidia.com/cuda-toolkit-archive这里选择要下载的版本
下载对应的离线run包
执行安装的时候要取消自带安装的530版本驱动，因为本地已经有驱动了。
注意这里cuda后缀标识代表会安装530.30.02版本的驱动，而我们本地已经安装了535的驱动，cuda版本是向下兼容的。
bash
root@xxx:~# chmod +x cuda_12.1.1_530.30.02_linux.run
# 安装完成
root@exai-121:~# sh cuda_12.1.1_530.30.02_linux.run
===========
= Summary =
===========
Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-12.1/
Please make sure that
- PATH includes /usr/local/cuda-12.1/bin
- LD_LIBRARY_PATH includes /usr/local/cuda-12.1/lib64, or, add /usr/local/cuda-12.1/lib64 to /etc/ld.so.conf and run ldconfig as root
To uninstall the CUDA Toolkit, run cuda-uninstaller in /usr/local/cuda-12.1/bin
***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 530.00 is required for CUDA 12.1 functionality to work.
To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file:
sudo <CudaInstaller>.run --silent --driver
Logfile is /var/log/cuda-installer.log
# 验证
root@xxx:~# /usr/local/cuda-12.1/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0ubuntu2204上不推荐使用530版本的驱动，安装会有一些依赖和GCC问题，尝试解决无果。推荐使用高版本驱动来向下兼容cuda版本。
问题：
make: *** [Makefile:82: modules] Error 2
-> Checking to see whether the nvidia kernel module was successfully built
executing: 'cd ./kernel; /usr/bin/make -k -j32 NV_EXCLUDE_KERNEL_MODULES="" SYSSRC="/lib/modules/6.5.0-41-generic/build" SYSOUT="/lib/modules/6.5.0-41-generic/build" NV_KERNEL_MODULES="nvidia"'...
make[1]: Entering directory '/usr/src/linux-headers-6.5.0-41-generic'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
You are using: cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Warning: Compiler version check failed:
The major and minor number of the compiler used to
compile the kernel:
x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38
does not match the compiler used here:
cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright © 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
It is recommended to set the CC environment variable
to the compiler that was used to compile the kernel.
To skip the test and silence this warning message, set
the IGNORE_CC_MISMATCH environment variable to "1".
However, mixing compiler versions between the kernel
and kernel modules can result in subtle bugs that are
difficult to diagnose.
Failed CC version check.
The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0 You are using: cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
make[3]: *** [scripts/Makefile.build:251: /tmp/selfgz2901/NVIDIA-Linux-x86_64-530.30.02/kernel/nvidia/i2c_nvswitch.o] Error 1
make[3]: Target '/tmp/selfgz2901/NVIDIA-Linux-x86_64-530.30.02/kernel/' not remade because of errors.
make[2]: *** [/usr/src/linux-headers-6.5.0-41-generic/Makefile:2039: /tmp/selfgz2901/NVIDIA-Linux-x86_64-530.30.02/kernel] Error 2
make[2]: Target 'modules' not remade because of errors.
make[1]: *** [Makefile:234: __sub-make] Error 2
make[1]: Target 'modules' not remade because of errors.
make[1]: Leaving directory '/usr/src/linux-headers-6.5.0-41-generic'
make: *** [Makefile:82: modules] Error 2
ERROR: The nvidia kernel module was not created.
ERROR: Installation has failed. Please see the file '/var/log/nvidia-installer.log' for details. You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com.
/tmp/selfgz2901/NVIDIA-Linux-x86_64-530.30.02/kernel/common/inc/nv-mm.h:88:60: warning: passing argument 4 of 'get_user_pages' makes pointer from integer without a cast [-Wint-conversion]
/tmp/selfgz2901/NVIDIA-Linux-x86_64-530.30.02/kernel/nvidia/nv-mmap.c:673:23: error: assignment of read-only member 'vm_flags'
尝试通过更换gcc版本解决：
bash
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12
update-alternatives --config gcc
# 弹出来的选项中选择12版本这是离线安装驱动方案下的问题，使用apt安装530版本未测试过，不确定包管理器是否会自动解决编译依赖问题
后续
使用sudo定制权限管理
https://blog.51cto.com/154773488/2449180
https://blog.csdn.net/Field_Yang/article/details/51547804
参考
https://www.cnblogs.com/tarsss/p/17433419.html
https://blog.csdn.net/jiexijihe945/article/details/131517630
cuda下载地址