docker 本地部署大模型(ollama)

docker 安装 ollama

复制代码
docker search ollama
docker pull ollama/ollama

###docker下载ollama部署
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

### 下载模型
docker exec -it ollama ollama pull llama3

### 交互式运行模型
 docker exec -it ollama ollama run llama3

注意有坑 :docker -v ollama:/root/.ollama

此处由于挂载目录使用了相对路径,所以本地文件夹位于 /var/lib/docker/volumes/ollama

而非运行命令的相对路径

测试 api

复制代码
$ curl http://localhost:14444/api/generate -d '{ "model": "llama3", "prompt": "請列出五樣台灣美食", "stream": true, "options": { "seed": 123, "top_k": 20, "top_p": 0.9, "temperature": 0 } }'
{"model":"llama3","created_at":"2024-07-08T03:49:16.611142384Z","response":"Here","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.758150085Z","response":" are","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.903960154Z","response":" ","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.048831268Z","response":"5","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.192653835Z","response":" must","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.340323632Z","response":"-","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.490090786Z","response":"try","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.630377981Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.79671948Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.947959672Z","response":":\n\n","done":false}
.......
{"model":"llama3","created_at":"2024-07-08T03:49:51.670524112Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.80466663Z","response":" showcase","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.953821627Z","response":" the","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.112141942Z","response":" diversity","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.274051058Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.41761068Z","response":" richness","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.571736412Z","response":" of","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.719734804Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.857012216Z","response":" cuisine","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.998817903Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.166612385Z","response":" which","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.342996684Z","response":" often","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.510572997Z","response":" combines","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.668246714Z","response":" Chinese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.836925764Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.986648055Z","response":" Japanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.13517455Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.283114108Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.443445171Z","response":" indigenous","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.603118958Z","response":" influences","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.76832015Z","response":".","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.931017326Z","response":" Enjoy","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.083288072Z","response":"!","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.533841685Z","response":" 🍴","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.671222985Z","response":"","done":true,"done_reason":"stop","context":[128006,882,128007,271,109311,32218,20834,76208,112595,117342,58666,102456,128009,128006,78191,128007,271,8586,527,220,20,2011,12,1568,94037,26863,1473,16,13,3146,3513,830,452,31707,61434,1754,96618,362,11670,94037,12269,11,25309,46895,273,19724,374,264,82651,323,94631,45993,1903,449,6435,23283,11059,25309,11,61260,11,323,24822,627,17,13,3146,46,22604,507,2727,1169,1754,96618,362,5526,8761,3691,304,29389,11,297,22604,297,2727,10145,527,73624,41951,19335,10409,449,628,1538,297,100163,11,70181,11,323,6307,45697,13,328,2841,449,264,10437,323,94760,19737,627,18,13,3146,626,39134,350,1073,84,1754,96618,49386,369,1202,281,2234,306,22843,11,357,39134,80247,374,264,96080,29839,17937,2027,430,596,5655,2269,4588,311,39143,13,1102,596,3629,10434,449,264,86921,19737,323,649,387,5115,57407,4999,19,13,3146,33,93613,85034,97685,1754,96618,362,69130,323,9257,12269,11,20716,4147,36167,36517,374,6435,23283,11059,304,264,21655,315,29839,19737,11,13465,11,323,49659,3156,279,13339,374,28682,323,17503,10980,6847,627,20,13,3146,38,4381,426,3524,1754,96618,7429,3967,439,94037,63452,11,1709,64,103253,527,4179,3690,293,11099,10409,449,20716,4147,36167,11,3820,839,24822,11,323,42393,13,362,18406,323,37154,40459,477,15496,382,9673,26863,35883,279,20057,323,90030,315,94037,36105,11,902,3629,33511,8620,11,11002,11,323,39678,34453,13,23485,0,11410,235,112,128009],"total_duration":43461066356,"load_duration":3482212558,"prompt_eval_count":17,"prompt_eval_duration":916597000,"eval_count":260,"eval_duration":39060126000}

python + ollama

py 复制代码
from ollama import Client 
client = Client(host=f'http://{IP_ADDRESS}:14444') 
response = client.chat(model='llama3', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ])
response 
复制代码
{'model': 'llama3',
 'created_at': '2024-07-08T04:05:41.013483539Z',
 'message': {'role': 'assistant',
  'content': "The sky appears blue because of a phenomenon called Rayleigh scattering, which is named after the British physicist Lord Rayleigh. Here's what happens:\n\n1. **Sunlight**: When sunlight enters Earth's atmosphere, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Molecules**: The atmosphere is made up of tiny molecules of gases like nitrogen (N2) and oxygen (O2). These molecules are much smaller than the wavelength of light.\n3. **Scattering**: When sunlight hits these small molecules, it scatters in all directions. This scattering effect is more pronounced for shorter wavelengths, like blue and violet light, which have a higher frequency than longer wavelengths, like red and orange light.\n4. **Blue dominance**: As a result of this scattering, the blue light with its shorter wavelength is dispersed throughout the atmosphere, reaching our eyes from all parts of the sky. This is why the sky appears blue during the daytime, especially when the sun is overhead.\n\nThe same principle applies to other colors, but with different intensities:\n\n* **Red and orange**: These longer wavelengths are less affected by scattering and reach our eyes more directly, which is why we see them as dominant at sunrise and sunset.\n* **Green and yellow**: These wavelengths are scattered similarly to blue light, but their intensity decreases with distance from the sun, making them less visible in the sky.\n\nSo, to summarize: the sky appears blue because of the scattering of sunlight by tiny molecules in the atmosphere, which favors shorter wavelengths like blue and violet."},
 'done_reason': 'stop',
 'done': True,
 'total_duration': 52984861518,
 'load_duration': 3257720984,
 'prompt_eval_count': 15,
 'prompt_eval_duration': 829317000,
 'eval_count': 325,
 'eval_duration': 4888172200

流式输出

py 复制代码
import ollama

stream = client.chat(
    model='llama3',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

参考

相关推荐
秋难降6 分钟前
Python 知识点详解(三)
python·编程语言
硬核隔壁老王14 分钟前
AI大模型从入门到精通系列教程(二):解锁Prompt Engineering——从原理到高阶技巧的AI交互指南
人工智能·程序员·llm
chao_7891 小时前
二分查找篇——寻找旋转排序数组中的最小值【LeetCode】
python·线性代数·算法·leetcode·矩阵
果子⌂1 小时前
Docker-构建镜像并实现LNMP架构
mysql·nginx·docker·容器·架构·php
无敌糖果1 小时前
K8S数据流核心底层逻辑剖析
docker·容器·kubernetes·k8s架构·底层逻辑·k8s数据流
金玉满堂@bj1 小时前
PyCharm 中 Python 解释器的添加选项及作用
ide·python·pycharm
程序员三藏1 小时前
如何使用Pytest进行测试?
自动化测试·软件测试·python·测试工具·职场和发展·测试用例·pytest
随心点儿2 小时前
使用python 将多个docx文件合并为一个word
开发语言·python·多个word合并为一个
不学无术の码农2 小时前
《Effective Python》第十三章 测试与调试——使用 Mock 测试具有复杂依赖的代码
开发语言·python
sleepybear11132 小时前
在Ubuntu上从零开始编译并运行Home Assistant源码并集成HACS与小米开源的Ha Xiaomi Home
python·智能家居·小米·home assistant·米家·ha xiaomi home