docker 本地部署大模型(ollama)

docker 安装 ollama

复制代码
docker search ollama
docker pull ollama/ollama

###docker下载ollama部署
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

### 下载模型
docker exec -it ollama ollama pull llama3

### 交互式运行模型
 docker exec -it ollama ollama run llama3

注意有坑 :docker -v ollama:/root/.ollama

此处由于挂载目录使用了相对路径,所以本地文件夹位于 /var/lib/docker/volumes/ollama

而非运行命令的相对路径

测试 api

复制代码
$ curl http://localhost:14444/api/generate -d '{ "model": "llama3", "prompt": "請列出五樣台灣美食", "stream": true, "options": { "seed": 123, "top_k": 20, "top_p": 0.9, "temperature": 0 } }'
{"model":"llama3","created_at":"2024-07-08T03:49:16.611142384Z","response":"Here","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.758150085Z","response":" are","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.903960154Z","response":" ","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.048831268Z","response":"5","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.192653835Z","response":" must","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.340323632Z","response":"-","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.490090786Z","response":"try","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.630377981Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.79671948Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.947959672Z","response":":\n\n","done":false}
.......
{"model":"llama3","created_at":"2024-07-08T03:49:51.670524112Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.80466663Z","response":" showcase","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.953821627Z","response":" the","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.112141942Z","response":" diversity","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.274051058Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.41761068Z","response":" richness","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.571736412Z","response":" of","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.719734804Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.857012216Z","response":" cuisine","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.998817903Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.166612385Z","response":" which","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.342996684Z","response":" often","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.510572997Z","response":" combines","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.668246714Z","response":" Chinese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.836925764Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.986648055Z","response":" Japanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.13517455Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.283114108Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.443445171Z","response":" indigenous","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.603118958Z","response":" influences","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.76832015Z","response":".","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.931017326Z","response":" Enjoy","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.083288072Z","response":"!","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.533841685Z","response":" 🍴","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.671222985Z","response":"","done":true,"done_reason":"stop","context":[128006,882,128007,271,109311,32218,20834,76208,112595,117342,58666,102456,128009,128006,78191,128007,271,8586,527,220,20,2011,12,1568,94037,26863,1473,16,13,3146,3513,830,452,31707,61434,1754,96618,362,11670,94037,12269,11,25309,46895,273,19724,374,264,82651,323,94631,45993,1903,449,6435,23283,11059,25309,11,61260,11,323,24822,627,17,13,3146,46,22604,507,2727,1169,1754,96618,362,5526,8761,3691,304,29389,11,297,22604,297,2727,10145,527,73624,41951,19335,10409,449,628,1538,297,100163,11,70181,11,323,6307,45697,13,328,2841,449,264,10437,323,94760,19737,627,18,13,3146,626,39134,350,1073,84,1754,96618,49386,369,1202,281,2234,306,22843,11,357,39134,80247,374,264,96080,29839,17937,2027,430,596,5655,2269,4588,311,39143,13,1102,596,3629,10434,449,264,86921,19737,323,649,387,5115,57407,4999,19,13,3146,33,93613,85034,97685,1754,96618,362,69130,323,9257,12269,11,20716,4147,36167,36517,374,6435,23283,11059,304,264,21655,315,29839,19737,11,13465,11,323,49659,3156,279,13339,374,28682,323,17503,10980,6847,627,20,13,3146,38,4381,426,3524,1754,96618,7429,3967,439,94037,63452,11,1709,64,103253,527,4179,3690,293,11099,10409,449,20716,4147,36167,11,3820,839,24822,11,323,42393,13,362,18406,323,37154,40459,477,15496,382,9673,26863,35883,279,20057,323,90030,315,94037,36105,11,902,3629,33511,8620,11,11002,11,323,39678,34453,13,23485,0,11410,235,112,128009],"total_duration":43461066356,"load_duration":3482212558,"prompt_eval_count":17,"prompt_eval_duration":916597000,"eval_count":260,"eval_duration":39060126000}

python + ollama

py 复制代码
from ollama import Client 
client = Client(host=f'http://{IP_ADDRESS}:14444') 
response = client.chat(model='llama3', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ])
response 
复制代码
{'model': 'llama3',
 'created_at': '2024-07-08T04:05:41.013483539Z',
 'message': {'role': 'assistant',
  'content': "The sky appears blue because of a phenomenon called Rayleigh scattering, which is named after the British physicist Lord Rayleigh. Here's what happens:\n\n1. **Sunlight**: When sunlight enters Earth's atmosphere, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Molecules**: The atmosphere is made up of tiny molecules of gases like nitrogen (N2) and oxygen (O2). These molecules are much smaller than the wavelength of light.\n3. **Scattering**: When sunlight hits these small molecules, it scatters in all directions. This scattering effect is more pronounced for shorter wavelengths, like blue and violet light, which have a higher frequency than longer wavelengths, like red and orange light.\n4. **Blue dominance**: As a result of this scattering, the blue light with its shorter wavelength is dispersed throughout the atmosphere, reaching our eyes from all parts of the sky. This is why the sky appears blue during the daytime, especially when the sun is overhead.\n\nThe same principle applies to other colors, but with different intensities:\n\n* **Red and orange**: These longer wavelengths are less affected by scattering and reach our eyes more directly, which is why we see them as dominant at sunrise and sunset.\n* **Green and yellow**: These wavelengths are scattered similarly to blue light, but their intensity decreases with distance from the sun, making them less visible in the sky.\n\nSo, to summarize: the sky appears blue because of the scattering of sunlight by tiny molecules in the atmosphere, which favors shorter wavelengths like blue and violet."},
 'done_reason': 'stop',
 'done': True,
 'total_duration': 52984861518,
 'load_duration': 3257720984,
 'prompt_eval_count': 15,
 'prompt_eval_duration': 829317000,
 'eval_count': 325,
 'eval_duration': 4888172200

流式输出

py 复制代码
import ollama

stream = client.chat(
    model='llama3',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

参考

相关推荐
apocelipes1 小时前
golang unique包和字符串内部化
java·python·性能优化·golang
Geoking.2 小时前
NumPy zeros() 函数详解
python·numpy
Full Stack Developme2 小时前
java.text 包详解
java·开发语言·python
丁浩6663 小时前
Python机器学习---2.算法:逻辑回归
python·算法·机器学习
tryCbest3 小时前
Linux使用Docker部署项目后期更新
linux·运维·docker
B站_计算机毕业设计之家3 小时前
计算机毕业设计:Python农业数据可视化分析系统 气象数据 农业生产 粮食数据 播种数据 爬虫 Django框架 天气数据 降水量(源码+文档)✅
大数据·爬虫·python·机器学习·信息可视化·课程设计·农业
Q_Q5110082853 小时前
python+uniapp基于微信小程序的旅游信息系统
spring boot·python·微信小程序·django·flask·uni-app·node.js
鄃鳕3 小时前
python迭代器解包【python】
开发语言·python
懷淰メ4 小时前
python3GUI--模仿百度网盘的本地文件管理器 By:PyQt5(详细分享)
开发语言·python·pyqt·文件管理·百度云·百度网盘·ui设计
Q_Q5110082854 小时前
python基于web的汽车班车车票管理系统/火车票预订系统/高铁预定系统 可在线选座
spring boot·python·django·flask·node.js·汽车·php