【ollama】(5):在本地使用docker-compose启动ollama镜像,修改模型存储位置,并下载qwen-0.5b模型,速度飞快

1,ollama项目

Ollama 是一个强大的框架,设计用于在 Docker 容器中部署 LLM。Ollama 的主要功能是在 Docker 容器内部署和管理 LLM 的促进者,它使该过程变得非常简单。它帮助用户快速在本地运行大模型,通过简单的安装指令,可以让用户执行一条命令就在本地运行开源大型语言模型,例如 Llama 2。

https://ollama.com/

https://www.bilibili.com/video/BV1HC411Y7P1/?vd_source=4b290247452adda4e56d84b659b0c8a2

【ollama】(5):在本地使用docker-compose启动ollama镜像,并下载qwen-0.5b模型,速度飞快

2,整个docker-compose 配置如下:

yaml 复制代码
version: '3.5'

services:

##################### 使用ollama部署大模型 #####################

# OLLAMA_HOST       The host:port to bind to (default "127.0.0.1:11434")
# OLLAMA_ORIGINS    A comma separated list of allowed origins.
# OLLAMA_MODELS     The path to the models directory (default is "~/.ollama/models")

  ollama:
    restart: always
    container_name: ollama
    image: ollama/ollama
    ports:
      - 8000:8000
    environment:
      - OLLAMA_HOST=0.0.0.0:8000
      - OLLAMA_MODELS=/data/models
    volumes:
      - ./models/:/data/models
    # 命令启动 serve
    command: serve

启动docker镜像服务:

bash 复制代码
docker-compose  up -d

然后就可以执行命令测试了:

bash 复制代码
curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
bash 复制代码
$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"很高兴"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为您"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"有什么"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"您的"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"吗"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"?"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}

data: [DONE]

速度特别快。还可以。

相关推荐
wno7041 小时前
CentOS下FTP服务搭建
linux·运维·centos
mylove_北极甜虾1 小时前
jenkins-01Jenkins 架构与任务调度
运维·jenkins
tianyuanwo14 小时前
PostgreSQL 部署运维实战总结:从安装踩坑到生产可用
运维·数据库·postgresql
Decay__15 小时前
SQLServer 优化案例集 —— 慢 SQL 优化、索引重构与参数调整收益对比
运维·数据库·sql·mysql·架构
优化Henry16 小时前
LTE 站点 S1 链路反复闪断导致小区频闪退服故障排查
运维·网络·学习·信息与通信·tdd
manageengine17 小时前
服务器 CPU 100% 怎么排查:8 条命令从现象定位到根因
运维·服务器·cpu
分布式存储与RustFS17 小时前
RustFS 分片上传怎么配:大文件断点续传的最小可验证流程
运维·云原生·开源·对象存储·分布式存储·s3·性能基准
爱吃香菜的初学者18 小时前
二十五.Linux——epoll
linux·运维·服务器
xbzb19 小时前
Kubernetes 节点删除后还有残留?drain、reset 与集群销毁重建全流程
容器·kubernetes·kubectl·集群·reset·drain
宽海智能仓储物流19 小时前
宽海智能WCS系统常见问题全解析:从选型对接、运维定制到降本增效
运维·数据仓库·自动化