prompt自主生成框架 - DSPy

DSPy是斯坦福大学搞的一个prompt自动生成框架，核心理念是

"Prompt不应该由人来写，应该由模型自己优化。"

目前已有很多项目组已经不在手写propmt，而是用DSPy自动生成。

这里尝试在本地环境运行DSPy，示例其prmopt的生成过程。

所用示例和代码，参考和修改自网络资料。

1 DSPy

1.1 DSPy说明

DSPy只需要定义任务逻辑，如"输入是问题，输出是答案"，然后准备高质量的数据集。

DSPy会自动在后台把这个Prompt"编译"出来。自动尝试各种Few-Shot的组合，调整指令，直到在测试集上效果达到最优。

1.2 DSPy安装

DSPy安装命令如下所示。

pip install dspy -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install dspy-ai -i https://pypi.tuna.tsinghua.edu.cn/simple

1.3 LLM安装

这里采用ollama大模型，并假设ollama模型gemma3n:e2b已安装。具体过程参考

https://blog.csdn.net/liliang199/article/details/152125310

2 DSPy测试

2.1 DSPy连接LLM

DSPy通过dspy.LM连接大模型，具体为通过model的名称区分。

比如"openai/gpt-4o", "ollama/gemma3n:e2b"

并通过api_base和token_key定义LLM的基础url和令牌。

这里通过ollma/gemma3n:e2b示例llm的调用过程，示例如下。

复制代码

import dspy


# Local Ollama
lm = dspy.LM(model="ollama/gemma3n:e2b", api_base="http://localhost:11434")

# Configure globally
dspy.configure(lm=lm)
responses = await lm.acall("What is the capital of USA?")
print(responses)

输出如下

$"The capital of the USA is \*\*Washington, D.C.\*\* \\n\\nIt's a federal district, not a state, and serves as the seat of the U.S. government.\\n\\n\\n\\n"$

如果是其他服务商提供了LLM，并且兼容openai接口，可以通过如是示例配置。

在模型名称前需要跟"openai"，示例如下

"openai/deepseek-r1"

openai兼容llm的配置示例

复制代码

import os
os.environ['OPENAI_API_KEY'] = "sk-xxxxxxxxxx"
os.environ['OPENAI_BASE_URL'] = "https://llm_provider_base_url/v1"
model_name = "openai/deepseek-r1"


import dspy

# OpenAI
lm = dspy.LM(model=model_name, temperature=0.7, max_tokens=1000)

# Configure globally
dspy.configure(lm=lm)

2.2 数学计算prompt示例

1）24点计算

任务示例如下，输入为

point_list="1,3,6,0"

需要通过加、减、乘、除、括号等基本运算符，将这4个数字组合成一个将表达式。

要求表达式的值为24，表达式只能包含这4个数字，并且这4个数必须出现在表达式中。

复制代码

point_list="1,3,6,0"

PROMPT_TEMPLATE = """
        Use the following four natural numbers {point_list} to form a four-rule operation expression. This expression calculates a value equal to 24, where only the four natural numbers and add, subtract, multiply, divide symbols and parentheses. The expression can only have four natural numbers given in the expression, and all four natural numbers must be included.
        Please simply return an expression that meets the above requirements, don't include '= 24'.
    """
question = PROMPT_TEMPLATE.format(point_list=f"[{point_list}]")
module = dspy.ChainOfThought("question -> answer")
result = module(question=question)
print(f"Hello111 ---{result}---")

输出如下所示。

DSPy经过一些列尝试，成功将1,3,6,0组合成一个表达式， answer='(6 + 0) * (3 + 1)。

经过LLM测试和实际计算，表达式结果为24，满足要求。

Hello111 ---Prediction(

reasoning="We need to find an expression using the numbers 1, 3, 6, and 0 with +, -, *, / and parentheses that equals 24. We must use all four numbers. Let's try different combinations.\n\nOne possible approach is to multiply some numbers and then combine them with other operations.\n- 6 * 3 * 1 = 18. We need to get to 24.\n- 6 * 3 = 18. We need to get to 24.\n- 18 * 1 = 18. We need to get to 24.\n- 18 + 6 = 24. This works!\n\nAnother approach:\n- 6 * 1 = 6. We need to get to 24.\n- 6 * 3 = 18. We need to get to 24.\n- 18 + 0 = 18. We need to get to 24.\n- 18 * 0 = 0. We need to get to 24.\n\nLet's try another combination:\n- 6 * 3 * 0 = 0. We need to get to 24.\n- 6 * 3 = 18. We need to get to 24.\n- 18 / 0 is undefined.\n\nLet's try:\n(6 - 3) * 1 * 0 = 3 * 1 * 0 = 0.\n\nLet's try:\n(6 + 3) * 1 * 0 = 9 * 1 * 0 = 0.\n\nLet's try:\n(6 * 3) * 1 * 0 = 18 * 1 * 0 = 0.\n\nLet's try:\n(6 + 0) * (3 - 1) = 6 * 2 = 12.\n\nLet's try:\n(6 + 0) * 3 * 1 = 6 * 3 * 1 = 18.\n\nLet's try:\n(6 + 0) * (3 + 1) = 6 * 4 = 24.",

answer='(6 + 0) * (3 + 1)'

)---

2）掷骰子计算

创建了一个 dspy.ChainOfThought 类型的问答模块 math，然后向 math 模块提出了一个问题：

掷出两个骰子，总和等于2的概率是多少？

示例代码如下

复制代码

math = dspy.ChainOfThought("question -> answer: float")
response = math(question="Two dice are tossed. What is the probability that the sum equals two?")
print(response)

机器上的运行结果如下。

Prediction(

reasoning='Let S be the sample space of the outcomes when two dice are tossed. Each die has 6 faces, so the total number of outcomes is $6 \\\\times 6 = 36$ .\nWe want to find the probability that the sum of the two dice is equal to 2. Let A be the event that the sum of the two dice is 2. The only outcome that satisfies this condition is (1, 1).\nThe number of outcomes in event A is 1.\nThe total number of outcomes in the sample space S is 36.\nThe probability of event A is the ratio of the number of outcomes in A to the total number of outcomes in S.\n $P(A) = \\\\frac{\\\\text{Number of outcomes in A}}{\\\\text{Total number of outcomes in S}} = \\\\frac{1}{36}$ .',

answer=0.02777777777777777

)

模型给醋的理由是

当两个骰子被掷出时，有36种可能的结果，因为每个骰子都有6个面。得到2之和的唯一方法是：两个骰子都显示一个1 (1+1)。因此，概率是1/36。

3）情感预测

以下是情感预测的示例代码。

复制代码

sentence = "it's a charming and often affecting journey."  # example from the SST-2 dataset.

# 1) Declare with a signature.
classify = dspy.Predict('sentence -> sentiment: bool')

# 2) Call with input argument(s). 
response = classify(sentence=sentence)

# 3) Access the output.
print(response.sentiment)

输出如下