【ollama】(5):在本地使用docker-compose启动ollama镜像,修改模型存储位置,并下载qwen-0.5b模型,速度飞快

1,ollama项目

Ollama 是一个强大的框架,设计用于在 Docker 容器中部署 LLM。Ollama 的主要功能是在 Docker 容器内部署和管理 LLM 的促进者,它使该过程变得非常简单。它帮助用户快速在本地运行大模型,通过简单的安装指令,可以让用户执行一条命令就在本地运行开源大型语言模型,例如 Llama 2。

https://ollama.com/

https://www.bilibili.com/video/BV1HC411Y7P1/?vd_source=4b290247452adda4e56d84b659b0c8a2

【ollama】(5):在本地使用docker-compose启动ollama镜像,并下载qwen-0.5b模型,速度飞快

2,整个docker-compose 配置如下:

yaml 复制代码
version: '3.5'

services:

##################### 使用ollama部署大模型 #####################

# OLLAMA_HOST       The host:port to bind to (default "127.0.0.1:11434")
# OLLAMA_ORIGINS    A comma separated list of allowed origins.
# OLLAMA_MODELS     The path to the models directory (default is "~/.ollama/models")

  ollama:
    restart: always
    container_name: ollama
    image: ollama/ollama
    ports:
      - 8000:8000
    environment:
      - OLLAMA_HOST=0.0.0.0:8000
      - OLLAMA_MODELS=/data/models
    volumes:
      - ./models/:/data/models
    # 命令启动 serve
    command: serve

启动docker镜像服务:

bash 复制代码
docker-compose  up -d

然后就可以执行命令测试了:

bash 复制代码
curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
bash 复制代码
$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"很高兴"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为您"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"有什么"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"您的"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"吗"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"?"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}

data: [DONE]

速度特别快。还可以。

相关推荐
Bruce_Liuxiaowei2 小时前
MQTT协议在物联网环境中的安全风险与防范指南
运维·网络·物联网·安全·网络安全
java之迷6 小时前
Windows环境下,源码启动+本地部署和启动开源项目Ragflow失败SRE模块
windows·docker·开源
-dzk-7 小时前
【3DGS复现】Autodl服务器复现3DGS《简单快速》《一次成功》《新手练习复现必备》
运维·服务器·python·计算机视觉·3d·三维重建·三维
CS Beginner7 小时前
【Linux】 Ubuntu 开发环境极速搭建
linux·运维·ubuntu
致宏Rex8 小时前
Docker 实战教程(7) | 镜像管理和仓库操作
运维·docker·容器
罗技1238 小时前
不用每次都改 `easysearch.yml` 也能改启动参数 —— 用 Docker 环境变量搞定一切
docker·容器·eureka
落日漫游10 小时前
Kubernetes容器运行时:cri-docker vs containerd
docker·kubernetes
wu~97010 小时前
web服务器有哪些?服务器和web服务器有什么区别
运维·服务器·前端
爱倒腾的老唐10 小时前
13、Linux 基本权限
linux·运维·服务器
罗政11 小时前
CentOS 7.6 系统源码部署 HivisionIDPhotos
linux·运维·centos