llamafactory 部署教程

bash 复制代码

conda create --name llamafactory python=3.10 --all

bash 复制代码

conda activate llamafactory

bash 复制代码

python --version

对应CUDA 12.4

bash 复制代码

pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124

PyTorch GPU 可用性

bash 复制代码

python -c "import torch; print(torch.__version__); print(torch.version.cuda); print(torch.cuda.is_available())"

下载对应版本的.whl文件

使用如下命令安装

bash 复制代码

pip install flash_attn-xxx.whl

bash 复制代码

git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -r requirements.txt

bash 复制代码

set CUDA_VISIBLE_DEVICES=0

llamafactory-cli webui

登录huggingface,获取access token, 选择read

bash 复制代码

hf auth login

黏贴token

set set HF_ENDPOINT= 

hf download FreedomIntelligence/medical-o1-reasoning-SFT --repo-type dataset --local-dir ./data

bash 复制代码

llamafactory-cli webui