使用Milvus和DeepSeek构建RAG demo

使用Milvus和DeepSeek构建RAG

准备数据

• 使用 Milvus 文档 2.4.x 中的 FAQ 页面作为我们 RAG 中的私有知识库，这是一个简单 RAG 管道的良好数据源。
• 下载 zip 文件并将文档解压到 milvus_docs 文件夹。

!wget https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip
!unzip -q milvus_docs_2.4.x_en.zip -d milvus_docs

--2025-10-11 03:35:17--  https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip
Resolving github.com (github.com)... 140.82.112.3
Connecting to github.com (github.com)|140.82.112.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: ......
2025-10-11 03:35:17 (13.4 MB/s) - 'milvus_docs_2.4.x_en.zip' saved [613094/613094]

​

从 milvus_docs/en/faq 文件夹加载所有 markdown 文件。对于每个文档，我们简单地使用 "# " 来分割文件中的内容，这样可以大致分离出 markdown 文件中每个主要部分的内容。

glob用于文件路径匹配，支持通配符模式

recursive=True允许递归搜索子目录

from glob import glob

text_lines = []

for file_path in glob("milvus_docs/en/faq/*.md", recursive=True):
    with open(file_path, "r") as file:
        file_text = file.read()

    text_lines += file_text.split("# ")

len(text_lines)

72

准备LLM和Embedding模型

定义一个 embedding 模型，使用 milvus_model 来生成文本嵌入。

!pip install "pymilvus[model]==2.5.10"

Collecting pymilvus==2.5.10 (from pymilvus[model]==2.5.10)
  Downloading pymilvus-2.5.10-py3-none-any.whl.metadata (5.7 kB)
Requirement already satisfied: setuptools>69 in /usr/local/lib/python3.11/dist-packages (from pymilvus==2.5.10->pymilvus[model]==2.5.10) (75.2.0)
Collecting grpcio<=1.67.1,>=1.49.1 (from pymilvus==2.5.10->pymilvus[model]==2.5.10)
  ......

from pymilvus import model as milvus_model

# OpenAI国内代理 https://api.apiyi.com/token
embedding_model = milvus_model.dense.OpenAIEmbeddingFunction(
    model_name = "text-embedding-3-large",
    api_key = "sk-@@@", # 使用了kaggle进行实验，为了安全此处直接隐藏了key
    base_url = "https://api.apiyi.com/v1",
    dimensions = 512
)

生成一个测试嵌入并打印其维度和前几个元素

test_embedding = embedding_model.encode_queries(["this is a test"])[0]
embedding_dim = len(test_embedding)
print(embedding_dim)
print(test_embedding[:10])

512
[-0.02768106  0.00601141 -0.01277587  0.08863478 -0.02640001 -0.0310568
 -0.03206086  0.13350613 -0.05238454  0.05885904]

test_embedding_0 = embedding_model.encode_queries(["this is a test"])[0]
print(test_embedding_0[:10])

[-0.02768106  0.00601141 -0.01277587  0.08863478 -0.02640001 -0.0310568
 -0.03206086  0.13350613 -0.05238454  0.05885904]

将数据加载到 Milvus

>>>快速入门 | Milvus 文档

创建 Collection

关于 MilvusClient 的参数：

• 将 uri 设置为本地文件，例如 ./milvus.db，是最方便的方法，因为它会自动利用 Milvus Lite 将所有数据存储在此文件中。
• 如果有大规模数据，可以在 Docker 或 Kubernetes 上设置性能更高的 Milvus 服务器。在此设置中，请使用服务器 URI，例如 http://localhost:19530，作为您的 uri。
• 如果想使用 Zilliz Cloud（Milvus 的完全托管云服务），请调整 uri 和 token，它们对应 Zilliz Cloud 中的 Public Endpoint 和 Api key。

from pymilvus import MilvusClient

milvus_client = MilvusClient(uri="./milvus_demo.db")
collection_name = "my_rag_collection"

检查 collection 是否已存在，如果存在则删除它

Collection 是一个二维表，具有固定的列和变化的行。每列代表一个字段，每行代表一个实体。要实现这样的结构化数据管理，需要一个 Schema。要插入的每个实体都必须符合 Schema 中定义的约束条件。

你可以确定 Collections 的方方面面，包括其 Schema、索引参数、度量类型，以及是否在创建时加载，以确保集合完全满足你的要求。

if milvus_client.has_collection(collection_name):
    milvus_client.drop_collection(collection_name)

创建一个具有指定参数的新 collection

如果我们不指定任何字段信息，Milvus 将自动创建一个默认的 id 字段作为主键，以及一个 vector 字段来存储向量数据。一个保留的 JSON 字段用于存储非 schema 定义的字段及其值。

• metric_type (距离度量类型): 作用：定义如何计算向量之间的相似程度。 例如：IP (内积) - 值越大通常越相似；L2 (欧氏距离) - 值越小越相似；COSINE (余弦相似度) - 通常转换为距离，值越小越相似。 选择依据：根据你的嵌入模型的特性和期望的相似性定义来选择。
• consistency_level (一致性级别): 作用：定义数据写入后，读取操作能多快看到这些新数据。 例如： Strong (强一致性): 总是读到最新数据，可能稍慢。 Bounded (有界过期): 可能读到几秒内旧数据，性能较好 (默认)。 Session (会话一致性): 自己写入的自己能立刻读到。 Eventually (最终一致性): 最终会读到新数据，但没时间保证，性能最好。 选择依据：在数据实时性要求和系统性能之间做权衡。

简单来说： metric_type：怎么算相似。 consistency_level：新数据多久能被读到。

milvus_client.create_collection(
    collection_name = collection_name,
    dimension = embedding_dim,
    metric_type = "IP", # 内积距离
    consistency_level = "Strong", 
    # 支持的值为 (`"Strong"`, `"Session"`, `"Bounded"`, `"Eventually"`)。
    # 更多详情请参见 https://milvus.io/docs/consistency.md#Consistency-Level。
)

插入数据

遍历文本行，创建嵌入，然后将数据插入 Milvus。

这里有一个新字段 text，它是在 collection schema 中未定义的字段。它将自动添加到保留的 JSON 动态字段中，该字段在高级别上可以被视为普通字段。

from tqdm import tqdm

data = []
doc_embeddings = embedding_model.encode_documents(text_lines)

len(doc_embeddings)

72

for i, line in enumerate(tqdm(text_lines, desc="Creating embeddings")):
    data.append({"id": i, "vector": doc_embeddings[i], "text": line})

milvus_client.insert(collection_name=collection_name, data=data)

Creating embeddings: 100%|██████████| 72/72 [00:00<00:00, 366492.58it/s]





{'insert_count': 72, 'ids': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71], 'cost': 0}

构建 RAG

检索查询数据

我们指定一个关于 Milvus 的常见问题

question = "How is data stored in milvus?"

在 collection 中搜索该问题，并检索语义上最匹配的前3个结果

search_res = milvus_client.search(
    collection_name=collection_name,
    data=embedding_model.encode_queries(
        [question]
    ), # 将问题转换为嵌入向量
    limit=3, # 返回前3个结果
    search_params={"metric_type": "IP", "params": {}},  # 内积距离
    output_fields=["text"],  # 返回text字段
)

看一下查询的搜索结果

import json

retrieved_lines_with_distances = [
   (res["entity"]["text"], res["distance"]) for res in search_res[0]
]
print(json.dumps(retrieved_lines_with_distances, indent=4))

[
    [
        " Where does Milvus store data?\n\nMilvus deals with two types of data, inserted data and metadata. \n\nInserted data, including vector data, scalar data, and collection-specific schema, are stored in persistent storage as incremental log. Milvus supports multiple object storage backends, including [MinIO](https://min.io/), [AWS S3](https://aws.amazon.com/s3/?nc1=h_ls), [Google Cloud Storage](https://cloud.google.com/storage?hl=en#object-storage-for-companies-of-all-sizes) (GCS), [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs), [Alibaba Cloud OSS](https://www.alibabacloud.com/product/object-storage-service), and [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) (COS).\n\nMetadata are generated within Milvus. Each Milvus module has its own metadata that are stored in etcd.\n\n###",
        0.7751177549362183
    ],
    [
        "How does Milvus handle vector data types and precision?\n\nMilvus supports Binary, Float32, Float16, and BFloat16 vector types.\n\n- Binary vectors: Store binary data as sequences of 0s and 1s, used in image processing and information retrieval.\n- Float32 vectors: Default storage with a precision of about 7 decimal digits. Even Float64 values are stored with Float32 precision, leading to potential precision loss upon retrieval.\n- Float16 and BFloat16 vectors: Offer reduced precision and memory usage. Float16 is suitable for applications with limited bandwidth and storage, while BFloat16 balances range and efficiency, commonly used in deep learning to reduce computational requirements without significantly impacting accuracy.\n\n###",
        0.643004298210144
    ],
    [
        "How does Milvus flush data?\n\nMilvus returns success when inserted data are loaded to the message queue. However, the data are not yet flushed to the disk. Then Milvus' data node writes the data in the message queue to persistent storage as incremental logs. If `flush()` is called, the data node is forced to write all data in the message queue to persistent storage immediately.\n\n###",
        0.6181490421295166
    ]
]

使用 LLM 获取 RAG 响应

将检索到的文档转换为字符串格式

context = "\n".join(
    [line_with_distance[0] for line_with_distance in retrieved_lines_with_distances]
)

context

" Where does Milvus store data?\n\nMilvus deals with two types of data, inserted data and metadata. \n\nInserted data, including vector data, scalar data, and collection-specific schema, are stored in persistent storage as incremental log. Milvus supports multiple object storage backends, including [MinIO](https://min.io/), [AWS S3](https://aws.amazon.com/s3/?nc1=h_ls), [Google Cloud Storage](https://cloud.google.com/storage?hl=en#object-storage-for-companies-of-all-sizes) (GCS), [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs), [Alibaba Cloud OSS](https://www.alibabacloud.com/product/object-storage-service), and [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) (COS).\n\nMetadata are generated within Milvus. Each Milvus module has its own metadata that are stored in etcd.\n\n###\nHow does Milvus handle vector data types and precision?\n\nMilvus supports Binary, Float32, Float16, and BFloat16 vector types.\n\n- Binary vectors: Store binary data as sequences of 0s and 1s, used in image processing and information retrieval.\n- Float32 vectors: Default storage with a precision of about 7 decimal digits. Even Float64 values are stored with Float32 precision, leading to potential precision loss upon retrieval.\n- Float16 and BFloat16 vectors: Offer reduced precision and memory usage. Float16 is suitable for applications with limited bandwidth and storage, while BFloat16 balances range and efficiency, commonly used in deep learning to reduce computational requirements without significantly impacting accuracy.\n\n###\nHow does Milvus flush data?\n\nMilvus returns success when inserted data are loaded to the message queue. However, the data are not yet flushed to the disk. Then Milvus' data node writes the data in the message queue to persistent storage as incremental logs. If `flush()` is called, the data node is forced to write all data in the message queue to persistent storage immediately.\n\n###"

question

'How is data stored in milvus?'

为语言模型定义系统和用户提示。

此提示是使用从 Milvus 检索到的文档组装而成的。

SYSTEM_PROMPT = """
Human: 你是一个 AI 助手。你能够从提供的上下文段落片段中找到问题的答案。
"""
USER_PROMPT = f"""
请使用以下用 <context> 标签括起来的信息片段来回答用 <question> 标签括起来的问题。最后追加原始回答的中文翻译，并用 <translated>和</translated> 标签标注。
<context>
{context}
</context>
<question>
{question}
</question>
<translated>
</translated>
"""

USER_PROMPT

"\n请使用以下用 <context> 标签括起来的信息片段来回答用 <question> 标签括起来的问题。最后追加原始回答的中文翻译，并用 <translated>和</translated> 标签标注。\n<context>\n Where does Milvus store data?\n\nMilvus deals with two types of data, inserted data and metadata. \n\nInserted data, including vector data, scalar data, and collection-specific schema, are stored in persistent storage as incremental log. Milvus supports multiple object storage backends, including [MinIO](https://min.io/), [AWS S3](https://aws.amazon.com/s3/?nc1=h_ls), [Google Cloud Storage](https://cloud.google.com/storage?hl=en#object-storage-for-companies-of-all-sizes) (GCS), [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs), [Alibaba Cloud OSS](https://www.alibabacloud.com/product/object-storage-service), and [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) (COS).\n\nMetadata are generated within Milvus. Each Milvus module has its own metadata that are stored in etcd.\n\n###\nHow does Milvus handle vector data types and precision?\n\nMilvus supports Binary, Float32, Float16, and BFloat16 vector types.\n\n- Binary vectors: Store binary data as sequences of 0s and 1s, used in image processing and information retrieval.\n- Float32 vectors: Default storage with a precision of about 7 decimal digits. Even Float64 values are stored with Float32 precision, leading to potential precision loss upon retrieval.\n- Float16 and BFloat16 vectors: Offer reduced precision and memory usage. Float16 is suitable for applications with limited bandwidth and storage, while BFloat16 balances range and efficiency, commonly used in deep learning to reduce computational requirements without significantly impacting accuracy.\n\n###\nHow does Milvus flush data?\n\nMilvus returns success when inserted data are loaded to the message queue. However, the data are not yet flushed to the disk. Then Milvus' data node writes the data in the message queue to persistent storage as incremental logs. If `flush()` is called, the data node is forced to write all data in the message queue to persistent storage immediately.\n\n###\n</context>\n<question>\nHow is data stored in milvus?\n</question>\n<translated>\n</translated>\n"

使用 DeepSeek 提供的 deepseek-chat 模型根据提示生成响应

from openai import OpenAI

deepseek_client = OpenAI(
    api_key = "sk-@@@", # 使用了kaggle进行实验，为了安全此处直接隐藏了key
    base_url = "https://api.deepseek.com/v1"         # DeepSeek API 的基地址
)

response = deepseek_client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": USER_PROMPT},
    ],
)

print(response.choices[0].message.content)

Milvus stores data in two categories: inserted data and metadata. Inserted data, including vector data, scalar data, and collection-specific schema, are stored as incremental logs in persistent object storage backends such as MinIO, AWS S3, Google Cloud Storage, Azure Blob Storage, Alibaba Cloud OSS, and Tencent Cloud Object Storage. Metadata generated within Milvus are stored in etcd.

<translated>
Milvus 将数据存储分为两类：插入的数据和元数据。插入的数据，包括向量数据、标量数据和集合特定模式，以增量日志的形式存储在持久化对象存储后端，例如 MinIO、AWS S3、Google Cloud Storage、Azure Blob Storage、阿里云 OSS 和腾讯云对象存储。Milvus 内部生成的元数据存储在 etcd 中。
</translated>