llama.cpp编译

llam.cpp编译

1. 下载&编译

git clone https://github.com/ggml-org/llama.cpp
cmake -S . -B build

2. 下载模型验证

# 下载地址
https://huggingface.co/filipealmeida/open-llama-7b-v2-open-instruct-GGUF/blob/main/ggml-model-Q4_0.gguf

# 验证
./llama-cli.exe -m .\models\7B\ggml-model-Q4_0.gguf -p "Tell me a joke." --n_predict 100