【ollama】(5):在本地使用docker-compose启动ollama镜像,修改模型存储位置,并下载qwen-0.5b模型,速度飞快

1,ollama项目

Ollama 是一个强大的框架,设计用于在 Docker 容器中部署 LLM。Ollama 的主要功能是在 Docker 容器内部署和管理 LLM 的促进者,它使该过程变得非常简单。它帮助用户快速在本地运行大模型,通过简单的安装指令,可以让用户执行一条命令就在本地运行开源大型语言模型,例如 Llama 2。

https://ollama.com/

https://www.bilibili.com/video/BV1HC411Y7P1/?vd_source=4b290247452adda4e56d84b659b0c8a2

【ollama】(5):在本地使用docker-compose启动ollama镜像,并下载qwen-0.5b模型,速度飞快

2,整个docker-compose 配置如下:

yaml 复制代码
version: '3.5'

services:

##################### 使用ollama部署大模型 #####################

# OLLAMA_HOST       The host:port to bind to (default "127.0.0.1:11434")
# OLLAMA_ORIGINS    A comma separated list of allowed origins.
# OLLAMA_MODELS     The path to the models directory (default is "~/.ollama/models")

  ollama:
    restart: always
    container_name: ollama
    image: ollama/ollama
    ports:
      - 8000:8000
    environment:
      - OLLAMA_HOST=0.0.0.0:8000
      - OLLAMA_MODELS=/data/models
    volumes:
      - ./models/:/data/models
    # 命令启动 serve
    command: serve

启动docker镜像服务:

bash 复制代码
docker-compose  up -d

然后就可以执行命令测试了:

bash 复制代码
curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
bash 复制代码
$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen:0.5b","stream":true,
        "messages": [
            {
                "role": "user",
                "content": "你好"
            }
        ]
    }'
data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"很高兴"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为您"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"有什么"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"您的"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"吗"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377122,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"?"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]}

data: {"id":"chatcmpl-163","object":"chat.completion.chunk","created":1710377123,"model":"qwen:0.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}

data: [DONE]

速度特别快。还可以。

相关推荐
sauTCc2 小时前
Docker初探
docker
宁zz2 小时前
乌班图安装jenkins
运维·jenkins
云上艺旅3 小时前
K8S学习之基础七十四:部署在线书店bookinfo
学习·云原生·容器·kubernetes
c无序3 小时前
【Docker-7】Docker是什么+Docker版本+Docker架构+Docker生态
docker·容器·架构
FixBug_Nick3 小时前
使用Docker安装及使用最新版本的Jenkins
docker·容器·jenkins
大丈夫立于天地间3 小时前
ISIS协议中的数据库同步
运维·网络·信息与通信
rainFFrain3 小时前
单例模式与线程安全
linux·运维·服务器·vscode·单例模式
ghostwritten5 小时前
Run Milvus in Kubernetes with Milvus Operator
容器·kubernetes·milvus
Zero_to_zero12345 小时前
解决docker的ubuntu系统中文乱码问题
ubuntu·docker·容器
@郭小茶5 小时前
docker-compose方式部署docker项目
运维·docker·容器