DSPy：LLM程序自动编译与提示词优化

一、引言

DSPy(Declarative Self-improving Python)由斯坦福NLP组推出，核心思想：用代码声明LLM程序的签名和模块，编译器自动优化提示词和Few-shot示例。告别手工Prompt Engineering！

二、签名声明

python 复制代码

import dspy

# 签名 = 输入/输出字段声明
class QA(dspy.Signature):
    """问答任务"""
    question = dspy.InputField(desc="用户的问题")
    context = dspy.InputField(desc="相关参考资料")
    answer = dspy.OutputField(desc="基于上下文的精确回答，不超过100字")

class Sentiment(dspy.Signature):
    """情感分析"""
    text = dspy.InputField()
    sentiment = dspy.OutputField(desc="positive, negative, or neutral")
    confidence = dspy.OutputField(desc="0-1之间的置信度")

class Summarize(dspy.Signature):
    """摘要"""
    document = dspy.InputField()
    summary = dspy.OutputField(desc="3句话以内的摘要")
    key_points = dspy.OutputField(desc="3个关键点，用列表格式")

三、模块组合

python 复制代码

# DSPy模块 = 可组合的LLM调用单元
# 内置: Predict, ChainOfThought, ReAct, ProgramOfThought

class RAG(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought(QA)  # CoT推理
    
    def forward(self, question):
        context = self.retrieve(question)
        return self.generate(question=question, context=context)

class MultiHopQA(dspy.Module):
    """多跳问答：需要多步推理"""
    def __init__(self):
        self.search = dspy.ChainOfThought("question -> search_query")
        self.retrieve = dspy.Retrieve(k=3)
        self.reason = dspy.ChainOfThought("context, question -> answer, evidence")
    
    def forward(self, question):
        # Step 1: 生成搜索查询
        query = self.search(question=question).search_query
        
        # Step 2: 检索
        docs = self.retrieve(query)
        
        # Step 3: 推理回答
        return self.reason(context=docs, question=question)

class ClassifierPipeline(dspy.Module):
    """多分类：分步处理"""
    def __init__(self):
        self.sentiment = dspy.Predict(Sentiment)
        self.topic = dspy.Predict("text -> topic")
    
    def forward(self, text):
        sent = self.sentiment(text=text)
        topic = self.topic(text=text)
        return dspy.Prediction(
            sentiment=sent.sentiment,
            confidence=sent.confidence,
            topic=topic.topic
        )

四、编译器自动优化

python 复制代码

# Teleprompter = 自动优化器
from dspy.teleprompt import BootstrapFewShot, COPRO, MIPRO

# 1. BootstrapFewShot: 自动选择Few-shot示例
teleprompter = BootstrapFewShot(
    metric=accuracy_metric,
    max_bootstrapped_demos=4,    # 最多4个示例
    max_rounds=3                  # 3轮自举
)

# 2. 编译: 给定训练集 → 自动优化Prompt+Demo
compiled_rag = teleprompter.compile(
    student=rag_program,
    trainset=trainset
)

# 3. MIPRO: 贝叶斯优化找最优提示词+示例
teleprompter = MIPRO(
    metric=accuracy_metric,
    num_candidates=10,
    init_temperature=1.0
)
compiled = teleprompter.compile(rag_program, trainset=trainset)

# 4. COPRO: 协作式Prompt优化
from dspy.teleprompt import COPRO
teleprompter = COPRO(
    metric=accuracy_metric,
    breadth=5,   # 每轮生成5个候选
    depth=3      # 3轮优化
)

五、评估

python 复制代码

from dspy.evaluate import Evaluate

# 评估器
evaluator = Evaluate(
    devset=testset,
    metric=accuracy_metric,
    num_threads=4,
    display_progress=True,
    display_table=True
)

# 评估编译前后的效果
baseline = evaluator(rag_program)
optimized = evaluator(compiled_rag)

print(f"Baseline: {baseline:.2%}")
print(f"Optimized: {optimized:.2%}")
print(f"Improvement: +{(optimized-baseline)*100:.1f}pp")

# 查看编译器自动生成的提示词
compiled_rag.save("optimized_rag.json")
# DSPy自动优化了哪些?
# 1. Few-shot示例选择(data-driven)
# 2. Prompt指令微调(auto-generated)
# 3. Chain-of-Thought步骤分解

六、DSPy vs 手工Prompt

阶段	手工Prompt	DSPy
开发	反复试提示词	声明签名
优化	手动调示例	编译器自动
评估	人工看几个case	全量自动评估
迭代	改prompt→看效果	加数据→重编译
多模型	每个模型改prompt	改LM配置即可

七、总结

DSPy范式转变：Prompt Engineer→Program Compiler。声明签名→定义模块→Teleprompter自动优化→评估。告别手工调Prompt，拥抱程序化LLM优化。