Meta-Llama-3-8B 部署

Meta-Llama-3-8B 模型文件地址

LLaMA-Factory 仓库地址

Download Ollama

conda create -n llama8b_ python==3.10 -y

conda activate llama8b

pip install -r requirements.txt -i https://pypi.mirrors.ustc.edu.cn/simple 

pip install modelscope -i https://pypi.mirrors.ustc.edu.cn/simple 

CUDA_VISIBLE_DEVICES=0 python src/web_demo.py \

--model_name_or_path /home/oneview/ai-test/model/Meta-Llama-3-8B-Instruct \

--template llama3 \

--infer_backend vllm \

--vllm_enforce_eager