docker 本地部署大模型(ollama)

docker 安装 ollama

复制代码
docker search ollama
docker pull ollama/ollama

###docker下载ollama部署
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

### 下载模型
docker exec -it ollama ollama pull llama3

### 交互式运行模型
 docker exec -it ollama ollama run llama3

注意有坑 :docker -v ollama:/root/.ollama

此处由于挂载目录使用了相对路径,所以本地文件夹位于 /var/lib/docker/volumes/ollama

而非运行命令的相对路径

测试 api

复制代码
$ curl http://localhost:14444/api/generate -d '{ "model": "llama3", "prompt": "請列出五樣台灣美食", "stream": true, "options": { "seed": 123, "top_k": 20, "top_p": 0.9, "temperature": 0 } }'
{"model":"llama3","created_at":"2024-07-08T03:49:16.611142384Z","response":"Here","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.758150085Z","response":" are","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.903960154Z","response":" ","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.048831268Z","response":"5","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.192653835Z","response":" must","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.340323632Z","response":"-","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.490090786Z","response":"try","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.630377981Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.79671948Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.947959672Z","response":":\n\n","done":false}
.......
{"model":"llama3","created_at":"2024-07-08T03:49:51.670524112Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.80466663Z","response":" showcase","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.953821627Z","response":" the","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.112141942Z","response":" diversity","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.274051058Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.41761068Z","response":" richness","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.571736412Z","response":" of","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.719734804Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.857012216Z","response":" cuisine","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.998817903Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.166612385Z","response":" which","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.342996684Z","response":" often","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.510572997Z","response":" combines","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.668246714Z","response":" Chinese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.836925764Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.986648055Z","response":" Japanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.13517455Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.283114108Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.443445171Z","response":" indigenous","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.603118958Z","response":" influences","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.76832015Z","response":".","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.931017326Z","response":" Enjoy","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.083288072Z","response":"!","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.533841685Z","response":" 🍴","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.671222985Z","response":"","done":true,"done_reason":"stop","context":[128006,882,128007,271,109311,32218,20834,76208,112595,117342,58666,102456,128009,128006,78191,128007,271,8586,527,220,20,2011,12,1568,94037,26863,1473,16,13,3146,3513,830,452,31707,61434,1754,96618,362,11670,94037,12269,11,25309,46895,273,19724,374,264,82651,323,94631,45993,1903,449,6435,23283,11059,25309,11,61260,11,323,24822,627,17,13,3146,46,22604,507,2727,1169,1754,96618,362,5526,8761,3691,304,29389,11,297,22604,297,2727,10145,527,73624,41951,19335,10409,449,628,1538,297,100163,11,70181,11,323,6307,45697,13,328,2841,449,264,10437,323,94760,19737,627,18,13,3146,626,39134,350,1073,84,1754,96618,49386,369,1202,281,2234,306,22843,11,357,39134,80247,374,264,96080,29839,17937,2027,430,596,5655,2269,4588,311,39143,13,1102,596,3629,10434,449,264,86921,19737,323,649,387,5115,57407,4999,19,13,3146,33,93613,85034,97685,1754,96618,362,69130,323,9257,12269,11,20716,4147,36167,36517,374,6435,23283,11059,304,264,21655,315,29839,19737,11,13465,11,323,49659,3156,279,13339,374,28682,323,17503,10980,6847,627,20,13,3146,38,4381,426,3524,1754,96618,7429,3967,439,94037,63452,11,1709,64,103253,527,4179,3690,293,11099,10409,449,20716,4147,36167,11,3820,839,24822,11,323,42393,13,362,18406,323,37154,40459,477,15496,382,9673,26863,35883,279,20057,323,90030,315,94037,36105,11,902,3629,33511,8620,11,11002,11,323,39678,34453,13,23485,0,11410,235,112,128009],"total_duration":43461066356,"load_duration":3482212558,"prompt_eval_count":17,"prompt_eval_duration":916597000,"eval_count":260,"eval_duration":39060126000}

python + ollama

py 复制代码
from ollama import Client 
client = Client(host=f'http://{IP_ADDRESS}:14444') 
response = client.chat(model='llama3', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ])
response 
复制代码
{'model': 'llama3',
 'created_at': '2024-07-08T04:05:41.013483539Z',
 'message': {'role': 'assistant',
  'content': "The sky appears blue because of a phenomenon called Rayleigh scattering, which is named after the British physicist Lord Rayleigh. Here's what happens:\n\n1. **Sunlight**: When sunlight enters Earth's atmosphere, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Molecules**: The atmosphere is made up of tiny molecules of gases like nitrogen (N2) and oxygen (O2). These molecules are much smaller than the wavelength of light.\n3. **Scattering**: When sunlight hits these small molecules, it scatters in all directions. This scattering effect is more pronounced for shorter wavelengths, like blue and violet light, which have a higher frequency than longer wavelengths, like red and orange light.\n4. **Blue dominance**: As a result of this scattering, the blue light with its shorter wavelength is dispersed throughout the atmosphere, reaching our eyes from all parts of the sky. This is why the sky appears blue during the daytime, especially when the sun is overhead.\n\nThe same principle applies to other colors, but with different intensities:\n\n* **Red and orange**: These longer wavelengths are less affected by scattering and reach our eyes more directly, which is why we see them as dominant at sunrise and sunset.\n* **Green and yellow**: These wavelengths are scattered similarly to blue light, but their intensity decreases with distance from the sun, making them less visible in the sky.\n\nSo, to summarize: the sky appears blue because of the scattering of sunlight by tiny molecules in the atmosphere, which favors shorter wavelengths like blue and violet."},
 'done_reason': 'stop',
 'done': True,
 'total_duration': 52984861518,
 'load_duration': 3257720984,
 'prompt_eval_count': 15,
 'prompt_eval_duration': 829317000,
 'eval_count': 325,
 'eval_duration': 4888172200

流式输出

py 复制代码
import ollama

stream = client.chat(
    model='llama3',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

参考

相关推荐
liliangcsdn2 分钟前
mac mlx大模型框架的安装和使用
java·前端·人工智能·python·macos
香蕉可乐荷包蛋6 分钟前
Python学习之路(十三)-常用函数的使用,及优化
开发语言·python·学习
惜.己14 分钟前
使用python的读取xml文件,简单的处理成元组数组
xml·开发语言·python·测试工具
倔强青铜三22 分钟前
苦练Python第25天:玩转字典
人工智能·python·面试
小周学学学32 分钟前
docker安装与简单项目上手
运维·docker·容器
倔强青铜三35 分钟前
苦练Python第23天:元组秘籍与妙用
人工智能·python·面试
hayson38 分钟前
langchaingo用法详解及源码解析(一)
langchain·llm
Norvyn_71 小时前
LeetCode|Day18|20. 有效的括号|Python刷题笔记
笔记·python·leetcode
chao_7891 小时前
更灵活方便的初始化、清除方法——fixture【pytest】
服务器·自动化测试·python·pytest
小刘|2 小时前
腾讯云服务上下载docker以及使用Rabbitmq的流程
docker·rabbitmq·腾讯云