第40章 Spring AI 实战：企业级 AI 应用架构
大白话：前面39章教你怎么造零件，这章教你怎么组装一辆能上路的车。用Spring AI把架构、缓存、降级、安全、监控、迭代全部串起来，搭一个真正能上线的AI系统。
40.1 为什么是 Spring AI
前面6个阶段的代码示例，Python 占了大多数。但如果你是 Java 后端团队------团队会的是 Spring Boot，运维的是 JVM，积累的是 Spring Cloud 生态------那你的 AI 应用开发主战场应该是 Spring AI。
Spring AI 是什么
Spring AI 是 Spring 官方推出的 AI 应用开发框架，2024年正式发布1.0正式版。它的定位很明确：让 Java 开发者用 Spring 的方式开发 AI 应用。
yaml
# 一行依赖搞定
dependencies:
- org.springframework.ai:spring-ai-openai-spring-boot-starter:1.0.0
- org.springframework.ai:spring-ai-pgvector-spring-boot-starter:1.0.0
- org.springframework.ai:spring-ai-rag:1.0.0Spring AI vs LangChain（Python）
| 对比维度 | Spring AI | LangChain (Python) |
|---|---|---|
| 语言生态 | Java / Spring Boot | Python |
| 上手成本 | 会Spring就会用，零学习成本 | 需要学Python+LangChain概念 |
| 企业级能力 | 天然集成Spring Security/Cloud/Micrometer | 需要额外搭 |
| 部署运维 | JVM成熟生态，运维团队熟悉 | Python部署+虚拟环境管理 |
| 社区生态 | 2024年起步，成长中 | 2023年爆发，生态最大 |
| 适合团队 | Java后端团队 | Python/全栈团队 |
| RAG支持 | 内置完整（向量库/Embedding/检索） | 内置完整 |
| Agent支持 | Function Calling原生支持 | 更灵活（LangGraph等） |
| 生产稳定性 | Spring生态背书，成熟稳定 | 需要自己搭基础设施 |
大白话：如果你团队是Java栈，Spring AI是当前最顺手的AI开发框架。它不是最灵活的，但是最"省心"的------Spring帮你把企业级该操心的安全、监控、配置、部署全部搞定了，你只需要关注AI业务逻辑。
Spring AI 核心能力一览
Spring AI 核心模块
├── Chat Model → 统一的对话模型抽象（OpenAI/Anthropic/Qwen/Ollama...）
├── Embedding Model → 向量化模型抽象（BGE/OpenAI/智谱...）
├── Vector Store → 向量数据库抽象（PgVector/Milvus/Chroma/Redis...）
├── RAG → 检索增强生成完整支持
├── Function Calling → 工具调用（原生Spring方法→AI工具）
├── Memory → 对话记忆（短期/长期）
├── Structured Output → 结构化输出（JSON→Java对象）
├── Evaluation → 模型评估
├── Observation → 可观测性（Micrometer集成）
└── Advisors → 拦截器模式（类似Spring AOP，可插拔中间件）大白话 ：Spring AI 把AI应用需要的所有零件都做成了Spring Bean------你注入一个ChatClient就像注入一个RestTemplate一样自然。配合Advisors拦截器，你可以像写Spring拦截器一样给AI加缓存、限流、安全检查、日志。
40.2 项目搭建：从零开始
40.2.1 项目结构
ai-enterprise-app/
├── pom.xml
├── src/main/java/com/example/ai/
│ ├── AiEnterpriseApplication.java # 启动类
│ ├── config/
│ │ ├── AiConfig.java # AI核心配置
│ │ ├── VectorStoreConfig.java # 向量库配置
│ │ ├── CacheConfig.java # 缓存配置
│ │ ├── SecurityConfig.java # 安全配置
│ │ └── ObservabilityConfig.java # 监控配置
│ ├── controller/
│ │ ├── ChatController.java # 对话接口
│ │ ├── RAGController.java # 知识库问答接口
│ │ ├── AgentController.java # Agent接口
│ │ └── DocumentController.java # 文档管理接口
│ ├── service/
│ │ ├── ChatService.java # 对话服务
│ │ ├── RagService.java # RAG服务
│ │ ├── AgentService.java # Agent服务
│ │ ├── CostControlService.java # 成本控制
│ │ ├── SecurityService.java # 安全防护
│ │ └── EvaluationService.java # 评估服务
│ ├── advisor/
│ │ ├── CacheAdvisor.java # 缓存拦截器
│ │ ├── CostAdvisor.java # 成本监控拦截器
│ │ ├── SecurityAdvisor.java # 安全拦截器
│ │ ├── ObservationAdvisor.java # 可观测性拦截器
│ │ └── FallbackAdvisor.java # 降级拦截器
│ ├── model/
│ │ ├── entity/ # 实体
│ │ ├── dto/ # DTO
│ │ └── enums/ # 枚举
│ ├── tool/ # Agent工具
│ │ ├── OrderTool.java
│ │ ├── WeatherTool.java
│ │ └── SearchTool.java
│ └── util/
│ ├── TokenCounter.java # Token计数
│ └── ResponseCache.java # 响应缓存
├── src/main/resources/
│ ├── application.yml
│ ├── prompts/ # Prompt模板
│ │ ├── chat.st
│ │ ├── rag.st
│ │ └── summary.st
│ └── db/migration/ # 数据库迁移
└── src/test/java/ # 测试40.2.2 Maven 依赖
xml
<dependencies>
<!-- Spring Boot 基础 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- Spring AI 核心 -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>
<!-- Spring AI 向量库 -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-pgvector-spring-boot-starter</artifactId>
</dependency>
<!-- Spring AI RAG -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-rag</artifactId>
</dependency>
<!-- Spring AI 可观测性 -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-observation</artifactId>
</dependency>
<!-- Micrometer 监控 -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp</artifactId>
</dependency>
<!-- 工具库 -->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-core</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>1.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>40.2.3 核心配置
yaml
spring:
application:
name: ai-enterprise-app
# ============ AI 模型配置 ============
ai:
openai:
# 主模型 - 路由策略支持多模型
api-key: ${OPENAI_API_KEY:sk-xxx}
base-url: ${OPENAI_BASE_URL:https://api.openai.com}
chat:
options:
model: gpt-4o-mini # 默认用小模型省钱
temperature: 0.7
max-tokens: 2000
# 备用模型配置
fallback:
model: gpt-4o
temperature: 0.7
# Embedding配置
embedding:
options:
model: text-embedding-3-small
# 向量库配置
vectorstore:
pgvector:
index-type: HNSW
distance-type: COSINE_DISTANCE
dimensions: 1536
# ============ 数据库配置 ============
datasource:
url: jdbc:postgresql://localhost:5432/ai_app
username: ${DB_USER:postgres}
password: ${DB_PASS:postgres}
jpa:
hibernate:
ddl-auto: validate
properties:
hibernate:
jdbc:
batch_size: 100 # 批量插入向量
flyway:
enabled: true
locations: classpath:db/migration
# ============ Redis 缓存 ============
data:
redis:
host: ${REDIS_HOST:localhost}
port: ${REDIS_PORT:6379}
timeout: 3s
lettuce:
pool:
max-active: 50
max-idle: 20
# ============ Caffeine 本地缓存 ============
cache:
type: redis
caffeine:
spec: maximumSize=10000,expireAfterWrite=300s
# ============ 监控 ============
management:
endpoints:
web:
exposure:
include: health,info,metrics,prometheus,env
metrics:
distribution:
percentiles-histogram:
http.server.requests: true
ai.chat.duration: true
tracing:
sampling:
probability: 1.0 # 生产环境设0.1
# ============ 自定义AI配置 ============
ai:
# 模型分级路由
routing:
enabled: true
rules:
- name: simple-query
condition: "message.length < 50 && !message.contains('分析')"
model: gpt-4o-mini
- name: complex-query
condition: "message.contains('分析') || message.contains('总结') || message.contains('报告')"
model: gpt-4o
- name: code-query
condition: "message.contains('代码') || message.contains('编程')"
model: gpt-4o
# 缓存配置
cache:
enabled: true
exact:
ttl: 3600 # 精确缓存1小时
semantic:
enabled: true
ttl: 86400 # 语义缓存24小时
threshold: 0.95 # 相似度阈值
# 成本控制
cost:
daily-budget-tokens: 1000000
alert-threshold: 0.8
fallback-threshold: 0.95
# 安全配置
security:
prompt-injection:
enabled: true
block-patterns:
- "ignore.*previous.*instruction"
- "system.*prompt"
- "reveal.*instructions"
content-moderation:
enabled: true
max-conversation-length: 20
# RAG配置
rag:
chunk-size: 400
chunk-overlap: 60
top-k: 5
rerank:
enabled: true
model: bge-reranker-v2-m340.3 核心配置类
40.3.1 AI 主配置
java
@Configuration
public class AiConfig {
/**
* 主对话客户端 - 默认走小模型
*/
@Bean
@Primary
public ChatClient chatClient(ChatClient.Builder builder) {
return builder
.defaultSystem("你是一个专业的AI助手。请基于提供的上下文准确回答问题。"
+ "如果上下文中没有相关信息，请明确说'我没有找到相关信息'，不要编造答案。"
+ "回答时请标注信息来源。")
.defaultAdvisors(
new SecurityAdvisor(), // 安全检查（最先执行）
new CacheAdvisor(), // 缓存
new CostAdvisor(), // 成本监控
new ObservationAdvisor() // 链路追踪
)
.build();
}
/**
* 高级模型客户端 - 复杂问题路由
*/
@Bean("advancedChatClient")
public ChatClient advancedChatClient(OpenAiChatModel chatModel) {
return ChatClient.builder(chatModel)
.defaultSystem("你是一个资深的分析专家。请进行深度分析，给出有洞察力的回答。")
.defaultAdvisors(
new SecurityAdvisor(),
new ObservationAdvisor()
)
.build();
}
/**
* Embedding模型
*/
@Bean
public EmbeddingModel embeddingModel(OpenAiEmbeddingModel model) {
return model;
}
/**
* 对话记忆 - 使用Redis持久化
*/
@Bean
public ChatMemory chatMemory(RedisChatMemoryRepository repository) {
return MessageWindowChatMemory.builder()
.maxMessages(20) // 滑动窗口
.chatMemoryRepository(repository)
.build();
}
/**
* 模型路由器 - 按问题复杂度路由到不同模型
*/
@Bean
public ModelRouter modelRouter(ChatClient defaultClient,
@Qualifier("advancedChatClient") ChatClient advancedClient) {
return new ModelRouter(defaultClient, advancedClient);
}
}40.3.2 模型路由器
java
/**
* 模型分级路由 - 简单问题走小模型，复杂问题走大模型
* 参见第35章成本控制
*/
public class ModelRouter {
private final ChatClient defaultClient; // gpt-4o-mini
private final ChatClient advancedClient; // gpt-4o
private final CacheManager cacheManager;
// 路由规则
private static final List<RoutingRule> RULES = List.of(
// 复杂问题 → 大模型
new RoutingRule("分析|总结|报告|对比|设计|方案|架构", "advanced", 0.8),
// 代码相关 → 大模型
new RoutingRule("代码|编程|实现|debug|bug|重构", "advanced", 0.9),
// 简单问答 → 小模型
new RoutingRule("你好|谢谢|再见|几点|天气", "default", 0.95)
);
/**
* 路由决策
*/
public String route(String userMessage) {
// 1. 检查缓存 - 路由决策也缓存
String cacheKey = "route:" + DigestUtils.md5Hex(userMessage);
String cached = cacheManager.getCache("routeCache").get(cacheKey, String.class);
if (cached != null) {
return cached;
}
// 2. 规则匹配
for (RoutingRule rule : RULES) {
if (rule.matches(userMessage)) {
cacheManager.getCache("routeCache").put(cacheKey, rule.model);
return rule.model;
}
}
// 3. 默认走小模型（省钱）
return "default";
}
/**
* 获取对应的ChatClient
*/
public ChatClient getClient(String route) {
return "advanced".equals(route) ? advancedClient : defaultClient;
}
/**
* 路由 + 调用一体化
*/
public String call(String userMessage, String conversationId) {
String route = route(userMessage);
ChatClient client = getClient(route);
log.info("路由决策: message='{}...', route={}, model={}",
userMessage.substring(0, Math.min(20, userMessage.length())), route,
route.equals("advanced") ? "gpt-4o" : "gpt-4o-mini");
return client.prompt()
.user(userMessage)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
.call()
.content();
}
record RoutingRule(String pattern, String model, double confidence) {
boolean matches(String message) {
return Pattern.compile(pattern, Pattern.CASE_INSENSITIVE)
.matcher(message).find();
}
}
}大白话：模型路由器就像医院的分诊台------感冒发烧去普通门诊（小模型），疑难杂症去专家门诊（大模型）。分对了，既省钱又高效。
40.4 Advisor 拦截器体系
Spring AI 的 Advisor 机制类似 Spring AOP，可以在AI调用前后插入自定义逻辑。这是实现缓存、安全、成本控制、监控的核心机制。
40.4.1 安全拦截器
java
/**
* 安全Advisor - 第34-37章安全防护的Spring AI实现
*/
public class SecurityAdvisor implements BaseAdvisor {
// Prompt注入检测模式
private static final List<Pattern> INJECTION_PATTERNS = List.of(
Pattern.compile("ignore.*previous.*instruction", Pattern.CASE_INSENSITIVE),
Pattern.compile("system.*prompt", Pattern.CASE_INSENSITIVE),
Pattern.compile("reveal.*instructions", Pattern.CASE_INSENSITIVE),
Pattern.compile("disregard.*above", Pattern.CASE_INSENSITIVE),
Pattern.compile("you.*are.*now.*a.*", Pattern.CASE_INSENSITIVE)
);
// 敏感信息模式（防止泄露PII）
private static final List<Pattern> PII_PATTERNS = List.of(
Pattern.compile("\\d{15,18}"), // 身份证
Pattern.compile("\\d{16,19}"), // 银行卡
Pattern.compile("1[3-9]\\d{9}") // 手机号
);
@Override
public AdvisedRequest before(AdvisedRequest request) {
String userMessage = request.userText();
// 1. Prompt注入检测
for (Pattern pattern : INJECTION_PATTERNS) {
if (pattern.matcher(userMessage).find()) {
log.warn("检测到Prompt注入尝试: {}", userMessage.substring(0, 50));
throw new SecurityException("检测到潜在的安全风险，请正常提问");
}
}
// 2. PII检测 - 脱敏
String sanitized = userMessage;
for (Pattern pattern : PII_PATTERNS) {
sanitized = pattern.matcher(sanitized).replaceAll("[已脱敏]");
}
if (!sanitized.equals(userMessage)) {
log.info("检测到敏感信息，已脱敏处理");
request = AdvisedRequest.from(request).userText(sanitized).build();
}
// 3. 输入长度限制（防Token炸弹）
if (userMessage.length() > 10000) {
throw new SecurityException("输入过长，请控制在10000字以内");
}
// 4. 标记隔离 - 用户输入用特殊标记包裹
String markedMessage = "<user_input>" + sanitized + "</user_input>";
request = AdvisedRequest.from(request).userText(markedMessage).build();
return request;
}
@Override
public AdvisedResponse after(AdvisedResponse response) {
String content = response.response().content();
// 输出安全检查 - 防止泄露系统信息
if (content.contains("System Prompt:") || content.contains("系统指令")) {
log.warn("检测到模型可能泄露系统信息");
return AdvisedResponse.from(response)
.response(Response.from("抱歉，我无法提供此类信息。"))
.build();
}
return response;
}
}40.4.2 缓存拦截器
java
/**
* 缓存Advisor - 精确缓存 + 语义缓存
* 参见第35章成本控制
*/
public class CacheAdvisor implements BaseAdvisor {
private final RedisTemplate<String, String> redisTemplate;
private final EmbeddingModel embeddingModel;
private final double semanticThreshold;
@Override
public AdvisedRequest before(AdvisedRequest request) {
// 缓存检查在before阶段做
String userMessage = request.userText();
String cacheKey = "ai:cache:" + DigestUtils.md5Hex(normalize(userMessage));
// 1. 精确缓存
String cached = redisTemplate.opsForValue().get(cacheKey);
if (cached != null) {
log.debug("精确缓存命中");
// 标记为缓存命中，跳过LLM调用
return AdvisedRequest.from(request)
.withCacheHit(cached)
.build();
}
// 2. 语义缓存
if (semanticThreshold > 0) {
float[] queryVector = embeddingModel.embed(userMessage);
String semanticKey = "ai:semantic:" + vectorToKey(queryVector);
// 用Redis向量搜索找相似问题
Set<String> similarKeys = redisTemplate.opsForSet()
.members(semanticKey);
if (similarKeys != null) {
for (String key : similarKeys) {
String similarity = redisTemplate.opsForValue()
.get(key + ":sim");
if (similarity != null
&& Double.parseDouble(similarity) >= semanticThreshold) {
String answer = redisTemplate.opsForValue().get(key + ":answer");
if (answer != null) {
log.debug("语义缓存命中, similarity={}", similarity);
return AdvisedRequest.from(request)
.withCacheHit(answer)
.build();
}
}
}
}
}
return request;
}
@Override
public AdvisedResponse after(AdvisedResponse response) {
// 缓存写入在after阶段做
String userMessage = response.request().userText();
String answer = response.response().content();
// 只缓存高质量回答（非错误、长度适中）
if (answer.length() > 10 && answer.length() < 5000) {
String cacheKey = "ai:cache:" + DigestUtils.md5Hex(normalize(userMessage));
// 精确缓存 - 1小时
redisTemplate.opsForValue().set(cacheKey, answer, Duration.ofHours(1));
// 语义缓存 - 24小时
if (semanticThreshold > 0) {
float[] vector = embeddingModel.embed(userMessage);
String semanticKey = "ai:semantic:" + vectorToKey(vector);
redisTemplate.opsForSet().add(semanticKey, cacheKey);
redisTemplate.opsForValue().set(cacheKey + ":answer", answer,
Duration.ofHours(24));
redisTemplate.opsForValue().set(cacheKey + ":sim", "1.0",
Duration.ofHours(24));
}
}
return response;
}
/**
* 文本归一化 - 去标点、统一大小写、去多余空格
*/
private String normalize(String text) {
return text.replaceAll("[\\p{Punct}\\s，。？！、；：""''（）【】]+", "")
.toLowerCase()
.trim();
}
private String vectorToKey(float[] vector) {
// 取前8维做粗略分桶（减少搜索范围）
return String.format("%.2f_%.2f_%.2f_%.2f",
vector[0], vector[1], vector[2], vector[3]);
}
}40.4.3 成本监控拦截器
java
/**
* 成本Advisor - Token计量 + 预算控制 + 自动降级
* 参见第35章成本控制
*/
public class CostAdvisor implements BaseAdvisor {
private final RedisTemplate<String, String> redisTemplate;
private final double dailyBudgetTokens;
private final double alertThreshold;
private final double fallbackThreshold;
// 模型价格表（每1K Token, 美元）
private static final Map<String, ModelPricing> PRICING = Map.of(
"gpt-4o-mini", new ModelPricing(0.00015, 0.0006),
"gpt-4o", new ModelPricing(0.0025, 0.01),
"gpt-4-turbo", new ModelPricing(0.01, 0.03)
);
@Override
public AdvisedRequest before(AdvisedRequest request) {
// 1. 检查今日Token用量
String today = LocalDate.now().toString();
String usageKey = "ai:tokens:" + today;
String usageStr = redisTemplate.opsForValue().get(usageKey);
double usedTokens = usageStr != null ? Double.parseDouble(usageStr) : 0;
double ratio = usedTokens / dailyBudgetTokens;
if (ratio >= fallbackThreshold) {
// 预算超限 - 降级到最便宜模型
log.warn("Token预算超限({}%), 强制降级", ratio * 100);
request = AdvisedRequest.from(request)
.withModel("gpt-4o-mini")
.build();
// 发告警
sendAlert("TOKEN_BUDGET_EXCEEDED", ratio);
} else if (ratio >= alertThreshold) {
// 预算告警
log.warn("Token预算告警({}%)", ratio * 100);
sendAlert("TOKEN_BUDGET_WARNING", ratio);
}
return request;
}
@Override
public AdvisedResponse after(AdvisedResponse response) {
// 记录Token消耗
Usage usage = response.response().usage();
if (usage != null) {
int inputTokens = usage.promptTokens();
int outputTokens = usage.completionTokens();
int totalTokens = usage.totalTokens();
// 按天累计
String today = LocalDate.now().toString();
String usageKey = "ai:tokens:" + today;
redisTemplate.opsForValue().increment(usageKey, totalTokens);
// 按模型累计
String model = response.request().model();
String modelKey = "ai:tokens:" + today + ":" + model;
redisTemplate.opsForValue().increment(modelKey, totalTokens);
// 按用户累计（分账）
String userId = response.request().context().get("userId");
if (userId != null) {
String userKey = "ai:tokens:" + today + ":user:" + userId;
redisTemplate.opsForValue().increment(userKey, totalTokens);
}
// 计算成本
ModelPricing pricing = PRICING.getOrDefault(model, PRICING.get("gpt-4o-mini"));
double cost = (inputTokens / 1000.0 * pricing.inputPrice())
+ (outputTokens / 1000.0 * pricing.outputPrice());
// 记录到监控
Metrics.counter("ai.tokens.total", "model", model).increment(totalTokens);
Metrics.counter("ai.cost.usd", "model", model).increment(cost);
Metrics.gauge("ai.budget.ratio", ratio);
log.info("Token消耗: input={}, output={}, model={}, cost=${}",
inputTokens, outputTokens, model, String.format("%.4f", cost));
}
return response;
}
private void sendAlert(String type, double ratio) {
// 实际场景：调用钉钉/企业微信/邮件
log.error("ALERT: {} - 预算使用率={}%", type, String.format("%.1f", ratio * 100));
}
record ModelPricing(double inputPrice, double outputPrice) {}
}40.4.4 可观测性拦截器
java
/**
* 可观测性Advisor - 链路追踪 + 指标记录
* 参见第38章可观测性
*/
public class ObservationAdvisor implements BaseAdvisor {
private final MeterRegistry meterRegistry;
private final Tracer tracer;
@Override
public AdvisedRequest before(AdvisedRequest request) {
// 开始Span
Span span = tracer.nextSpan().name("ai.chat.request").start();
span.tag("ai.model", request.model());
span.tag("ai.user.id", request.context().getOrDefault("userId", "anonymous"));
span.tag("ai.conversation.id",
request.context().getOrDefault("conversationId", "unknown"));
span.tag("ai.input.length", String.valueOf(request.userText().length()));
// 把Span存到上下文
request = AdvisedRequest.from(request)
.withContext(Map.of("trace.span", span))
.build();
// 计时开始
long startTime = System.currentTimeMillis();
request.context().put("startTime", startTime);
return request;
}
@Override
public AdvisedResponse after(AdvisedResponse response) {
long startTime = (long) response.request().context().get("startTime");
long duration = System.currentTimeMillis() - startTime;
// 结束Span
Span span = (Span) response.request().context().get("trace.span");
if (span != null) {
Usage usage = response.response().usage();
if (usage != null) {
span.tag("ai.tokens.input", String.valueOf(usage.promptTokens()));
span.tag("ai.tokens.output", String.valueOf(usage.completionTokens()));
}
span.tag("ai.duration.ms", String.valueOf(duration));
span.tag("ai.cache.hit",
String.valueOf(response.request().context().getOrDefault("cacheHit", false)));
span.end();
}
// 记录指标
meterRegistry.timer("ai.chat.duration",
"model", response.request().model(),
"status", "success"
).record(Duration.ofMillis(duration));
if (response.request().context().getOrDefault("cacheHit", false)) {
meterRegistry.counter("ai.cache.hits").increment();
} else {
meterRegistry.counter("ai.cache.misses").increment();
}
return response;
}
}大白话：四个Advisor就像流水线上的四道质检工序------安全先检查（有没有危险品）、缓存先看看（有没有现货）、成本先算算（超预算没）、全程记录（出问题能追溯）。每一道都是独立可插拔的，不需要了直接去掉就行。
40.5 对话服务
40.5.1 ChatService
java
@Service
public class ChatService {
private final ModelRouter modelRouter;
private final ChatMemory chatMemory;
private final RedisTemplate<String, String> redisTemplate;
/**
* 普通对话 - 带模型路由 + 记忆
*/
public ChatResponse chat(String userId, String conversationId, String message) {
// 1. 记录对话历史
chatMemory.add(conversationId, new UserMessage(message));
// 2. 模型路由
String route = modelRouter.route(message);
ChatClient client = modelRouter.getClient(route);
// 3. 调用（Advisor链自动执行安全/缓存/成本/监控）
String response = client.prompt()
.messages(chatMemory.get(conversationId))
.user(message)
.advisors(a -> a
.param(ChatMemory.CONVERSATION_ID, conversationId)
.param("userId", userId)
)
.call()
.content();
// 4. 记录AI回复
chatMemory.add(conversationId, new AssistantMessage(response));
// 5. 记录到持久化存储（用于审计和后续分析）
saveChatHistory(userId, conversationId, message, response, route);
return ChatResponse.builder()
.content(response)
.model(route.equals("advanced") ? "gpt-4o" : "gpt-4o-mini")
.conversationId(conversationId)
.timestamp(LocalDateTime.now())
.build();
}
/**
* 流式对话 - SSE
*/
public Flux<String> chatStream(String userId, String conversationId, String message) {
chatMemory.add(conversationId, new UserMessage(message));
String route = modelRouter.route(message);
ChatClient client = modelRouter.getClient(route);
return client.prompt()
.messages(chatMemory.get(conversationId))
.user(message)
.advisors(a -> a
.param(ChatMemory.CONVERSATION_ID, conversationId)
.param("userId", userId)
)
.stream()
.content()
.doOnComplete(() -> {
// 流式完成后记录完整回复
chatMemory.add(conversationId,
new AssistantMessage("[streamed response]"));
log.info("流式对话完成: user={}, conv={}", userId, conversationId);
});
}
/**
* 多轮对话上下文摘要 - 防止上下文过长
*/
@Scheduled(fixedRate = 300000) // 每5分钟检查一次
public void compressConversationContext() {
// 找出消息超过15条的会话
Set<String> activeConversations = redisTemplate.opsForSet()
.members("ai:active:conversations");
if (activeConversations == null) return;
for (String convId : activeConversations) {
List<Message> messages = chatMemory.get(convId);
if (messages.size() > 15) {
// 摘要前10条消息
String summary = summarizeMessages(messages.subList(0, 10));
// 保留摘要 + 最近5条
chatMemory.clear(convId);
chatMemory.add(convId, new SystemMessage("之前的对话摘要: " + summary));
messages.subList(messages.size() - 5, messages.size())
.forEach(msg -> chatMemory.add(convId, msg));
log.info("压缩对话上下文: conv={}, {}条→{}条", convId,
messages.size(), 6);
}
}
}
private String summarizeMessages(List<Message> messages) {
String dialog = messages.stream()
.map(m -> m.getMessageType() == MessageType.USER
? "用户: " + m.getContent()
: "AI: " + m.getContent())
.collect(Collectors.joining("\n"));
return modelRouter.getClient("default")
.prompt()
.system("请将以下对话摘要为200字以内的总结，保留关键信息和上下文。")
.user(dialog)
.call()
.content();
}
private void saveChatHistory(String userId, String convId,
String message, String response, String route) {
// 持久化到数据库，用于审计和后续优化
// 实际场景：异步写入，不影响响应延迟
CompletableFuture.runAsync(() -> {
// chatHistoryRepository.save(...)
});
}
}40.6 RAG 知识库服务
40.6.1 文档处理与入库
java
@Service
public class RagService {
private final VectorStore vectorStore; // Spring AI统一向量库抽象
private final EmbeddingModel embeddingModel;
private final ChatClient chatClient;
/**
* 文档上传 - 切分 + 向量化 + 入库
*/
@Async
public CompletableFuture<Integer> ingestDocument(MultipartFile file,
String category,
String source) {
// 1. 加载文档
List<Document> documents = loadDocument(file);
// 2. 数据清洗（参见第22章）
documents = documents.stream()
.map(this::cleanDocument)
.filter(d -> !d.content.trim().isEmpty())
.collect(Collectors.toList());
// 3. 切分 - 递归切分，chunk_size=400, overlap=60
List<Document> chunks = splitDocuments(documents, 400, 60);
// 4. 添加元数据
chunks = chunks.stream()
.map(chunk -> Document.from(chunk.content)
.withMetadata(Map.of(
"category", category,
"source", source,
"fileName", file.getOriginalFilename(),
"fileSize", String.valueOf(file.getSize()),
"uploadTime", LocalDateTime.now().toString(),
"chunkIndex", String.valueOf(chunk.metadata.get("chunkIndex")),
"totalChunks", String.valueOf(chunk.metadata.get("totalChunks"))
))
)
.collect(Collectors.toList());
// 5. 向量化 + 入库（Spring AI一行搞定）
vectorStore.add(chunks);
log.info("文档入库完成: file={}, chunks={}, category={}",
file.getOriginalFilename(), chunks.size(), category);
return CompletableFuture.completedFuture(chunks.size());
}
/**
* RAG问答 - 检索 + 生成
*/
public RagResponse query(String userId, String question, String conversationId) {
long startTime = System.currentTimeMillis();
// 1. 查询改写（多轮对话场景）
String rewrittenQuery = rewriteQuery(question, conversationId);
// 2. 元数据过滤（按权限/类别）
Expression expression = new Expression("category == '" + getUserCategory(userId) + "'");
// 3. 向量检索（Spring AI内置）
List<Document> retrieved = vectorStore.similaritySearch(
SearchRequest.query(rewrittenQuery)
.topK(5)
.similarityThreshold(0.7)
.filterExpression(expression)
);
// 4. 检查检索结果
if (retrieved.isEmpty()) {
return RagResponse.builder()
.answer("抱歉，在知识库中没有找到与您问题相关的内容。"
+ "您可以尝试换个问法，或联系人工客服。")
.hasContext(false)
.build();
}
// 5. 组装上下文（参见第26章生成优化）
String context = buildContext(retrieved);
// 6. 生成回答（带引用标注）
String answer = chatClient.prompt()
.system(ragSystemPrompt())
.user(buildRagPrompt(question, context, retrieved))
.advisors(a -> a.param("userId", userId)
.param("conversationId", conversationId))
.call()
.content();
// 7. 幻觉检测（参见第26章）
boolean hallucinationCheck = checkHallucination(answer, context);
// 8. 构建引用
List<RagResponse.Citation> citations = buildCitations(retrieved);
long duration = System.currentTimeMillis() - startTime;
return RagResponse.builder()
.answer(answer)
.hasContext(true)
.citations(citations)
.retrievedCount(retrieved.size())
.hallucinationWarning(!hallucinationCheck)
.durationMs(duration)
.build();
}
/**
* RAG流式问答
*/
public Flux<String> queryStream(String userId, String question,
String conversationId) {
String rewritten = rewriteQuery(question, conversationId);
List<Document> retrieved = vectorStore.similaritySearch(
SearchRequest.query(rewritten).topK(5).similarityThreshold(0.7));
if (retrieved.isEmpty()) {
return Flux.just("抱歉，知识库中没有找到相关内容。");
}
String context = buildContext(retrieved);
return chatClient.prompt()
.system(ragSystemPrompt())
.user(buildRagPrompt(question, context, retrieved))
.stream()
.content();
}
/**
* RAG系统Prompt
*/
private String ragSystemPrompt() {
return """
你是一个专业的知识库问答助手。请严格基于以下检索到的上下文回答问题。
规则：
1. 只使用<context>中的信息回答问题，不要使用外部知识
2. 如果上下文中没有相关信息，明确说"根据知识库，我没有找到相关信息"
3. 回答时标注引用来源，格式：[来源:文件名]
4. 不要编造、猜测或推断上下文中没有的信息
5. 保持回答准确、简洁、有条理
""";
}
private String buildRagPrompt(String question, String context,
List<Document> documents) {
StringBuilder sb = new StringBuilder();
sb.append("<context>\n");
for (int i = 0; i < documents.size(); i++) {
sb.append(String.format("[来源%d: %s]\n%s\n\n",
i + 1,
documents.get(i).metadata.get("fileName"),
documents.get(i).content));
}
sb.append("</context>\n\n");
sb.append("<question>").append(question).append("</question>\n");
sb.append("请基于上述上下文回答问题，标注引用来源。");
return sb.toString();
}
// ... 省略辅助方法（loadDocument/cleanDocument/splitDocuments等）
}40.6.2 Controller
java
@RestController
@RequestMapping("/api/ai")
public class ChatController {
private final ChatService chatService;
private final RagService ragService;
/**
* 普通对话
*/
@PostMapping("/chat")
public ChatResponse chat(@RequestBody ChatRequest request,
@RequestHeader("X-User-Id") String userId) {
return chatService.chat(userId, request.conversationId(), request.message());
}
/**
* 流式对话 - SSE
*/
@PostMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> chatStream(@RequestBody ChatRequest request,
@RequestHeader("X-User-Id") String userId) {
return chatService.chatStream(userId, request.conversationId(), request.message());
}
/**
* RAG知识库问答
*/
@PostMapping("/rag/query")
public RagResponse ragQuery(@RequestBody RagRequest request,
@RequestHeader("X-User-Id") String userId) {
return ragService.query(userId, request.question(), request.conversationId());
}
/**
* RAG流式问答
*/
@PostMapping(value = "/rag/query/stream",
produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> ragQueryStream(@RequestBody RagRequest request,
@RequestHeader("X-User-Id") String userId) {
return ragService.queryStream(userId, request.question(),
request.conversationId());
}
/**
* 文档上传
*/
@PostMapping("/documents/upload")
public CompletableFuture<UploadResponse> uploadDocument(
@RequestParam("file") MultipartFile file,
@RequestParam("category") String category,
@RequestHeader("X-User-Id") String userId) {
return ragService.ingestDocument(file, category, userId)
.thenApply(chunks -> new UploadResponse(true,
"文档入库成功，共" + chunks + "个片段", chunks));
}
/**
* 健康检查
*/
@GetMapping("/health")
public Map<String, Object> health() {
return Map.of(
"status", "UP",
"timestamp", LocalDateTime.now(),
"models", Map.of("default", "gpt-4o-mini", "advanced", "gpt-4o")
);
}
}40.7 Agent 工具集成
40.7.1 工具定义
Spring AI 的 Function Calling 非常简洁------一个 @Bean 就是一个工具：
java
@Configuration
public class ToolConfig {
/**
* 工具1：查询订单
*/
@Bean
@Description("根据订单号查询订单状态、物流信息和商品详情")
public Function<QueryOrderRequest, QueryOrderResponse> queryOrder() {
return request -> {
log.info("Agent调用queryOrder工具: orderId={}", request.orderId());
// 实际查询逻辑
Order order = orderRepository.findById(request.orderId())
.orElseThrow(() -> new RuntimeException("订单不存在"));
return new QueryOrderResponse(
order.getId(),
order.getStatus().toString(),
order.getTrackingNo(),
order.getEstimatedDelivery().toString(),
order.getItems().stream()
.map(item -> item.getName() + " x" + item.getQuantity())
.collect(Collectors.toList())
);
};
}
/**
* 工具2：申请退款
*/
@Bean
@Description("为指定订单申请退款。仅在订单状态为'已签收'且签收不超过7天时可用。")
public Function<RefundRequest, RefundResponse> applyRefund() {
return request -> {
// 权限检查 + 业务校验
Order order = orderRepository.findById(request.orderId())
.orElseThrow(() -> new RuntimeException("订单不存在"));
if (order.getStatus() != OrderStatus.DELIVERED) {
return new RefundResponse(false, "订单状态不支持退款");
}
if (order.getDeliveredAt().isBefore(LocalDateTime.now().minusDays(7))) {
return new RefundResponse(false, "已超过7天退款期限");
}
// 创建退款单
Refund refund = refundService.createRefund(order, request.reason());
return new RefundResponse(true, "退款申请已创建",
refund.getId(), refund.getStatus().toString());
};
}
/**
* 工具3：搜索知识库
*/
@Bean
@Description("在企业知识库中搜索相关信息，返回相关文档片段")
public Function<SearchKnowledgeRequest, SearchKnowledgeResponse> searchKnowledge() {
return request -> {
List<Document> results = vectorStore.similaritySearch(
SearchRequest.query(request.query())
.topK(3)
.similarityThreshold(0.7)
);
return new SearchKnowledgeResponse(
results.stream()
.map(doc -> Map.of(
"content", doc.content,
"source", doc.metadata.get("fileName")
))
.collect(Collectors.toList())
);
};
}
// 请求/响应记录
public record QueryOrderRequest(String orderId) {}
public record QueryOrderResponse(String orderId, String status,
String trackingNo, String estimatedDelivery, List<String> items) {}
public record RefundRequest(String orderId, String reason) {}
public record RefundResponse(boolean success, String message,
String refundId, String status) {}
public record SearchKnowledgeRequest(String query) {}
public record SearchKnowledgeResponse(List<Map<String, Object>> results) {}
}40.7.2 Agent 服务
java
@Service
public class AgentService {
private final ChatClient chatClient;
private final ChatMemory chatMemory;
/**
* Agent对话 - 自动决策使用工具
*/
public AgentResponse execute(String userId, String conversationId,
String task) {
chatMemory.add(conversationId, new UserMessage(task));
// Spring AI自动处理Function Calling循环
String response = chatClient.prompt()
.system("""
你是一个智能客服Agent。你可以使用提供的工具帮助用户解决问题。
规则：
1. 先理解用户需求，决定是否需要调用工具
2. 调用工具后，基于工具返回的结果回答用户
3. 如果工具返回失败，向用户解释原因并提供替代方案
4. 高风险操作（如退款）必须向用户确认后才执行
5. 不要编造订单信息，只使用工具查询到的真实数据
""")
.user(task)
.functions("queryOrder", "applyRefund", "searchKnowledge")
.advisors(a -> a
.param(ChatMemory.CONVERSATION_ID, conversationId)
.param("userId", userId)
)
.call()
.content();
chatMemory.add(conversationId, new AssistantMessage(response));
return AgentResponse.builder()
.content(response)
.conversationId(conversationId)
.build();
}
/**
* Agent流式执行
*/
public Flux<String> executeStream(String userId, String conversationId,
String task) {
return chatClient.prompt()
.system("你是一个智能客服Agent...")
.user(task)
.functions("queryOrder", "applyRefund", "searchKnowledge")
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
.stream()
.content();
}
}大白话 ：Spring AI 的 Function Calling 简单到什么程度？你写一个普通的 Spring Bean 方法，加个 @Description 注解告诉模型这个工具干什么，Spring AI 自动帮你处理"模型决定调用→生成参数→执行函数→结果返回模型→模型基于结果回答"的完整循环。你不需要写任何调度代码。
40.8 安全配置
40.8.1 Spring Security + AI 安全
java
@Configuration
@EnableWebSecurity
public class SecurityConfig {
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
return http
// 1. 认证配置
.authorizeHttpRequests(auth -> auth
.requestMatchers("/api/ai/health").permitAll()
.requestMatchers("/api/ai/**").authenticated()
.anyRequest().authenticated()
)
// 2. JWT认证
.oauth2ResourceServer(oauth -> oauth.jwt(Customizer.withDefaults()))
// 3. 限流
.addFilterBefore(new RateLimitFilter(), UsernamePasswordAuthenticationFilter.class)
// 4. AI安全过滤
.addFilterAfter(new AiSecurityFilter(), RateLimitFilter.class)
// 5. CORS
.cors(Customizer.withDefaults())
// 6. CSRF（API场景关闭）
.csrf(csrf -> csrf.disable())
.build();
}
/**
* 限流过滤器 - 每用户每分钟20次
*/
static class RateLimitFilter extends OncePerRequestFilter {
private final RedisTemplate<String, String> redisTemplate;
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
String userId = request.getHeader("X-User-Id");
String key = "ratelimit:" + userId + ":" + System.currentTimeMillis() / 60000;
Long count = redisTemplate.opsForValue().increment(key);
if (count == 1) {
redisTemplate.expire(key, Duration.ofMinutes(1));
}
if (count > 20) {
response.setStatus(429);
response.getWriter().write("{\"error\":\"请求过于频繁，请稍后再试\"}");
return;
}
chain.doFilter(request, response);
}
}
/**
* AI安全过滤器 - Prompt注入检测
*/
static class AiSecurityFilter extends OncePerRequestFilter {
private static final List<Pattern> INJECTION_PATTERNS = List.of(
// 参见第37章完整模式列表
Pattern.compile("ignore.*previous.*instruction", Pattern.CASE_INSENSITIVE),
Pattern.compile("system.*prompt", Pattern.CASE_INSENSITIVE)
);
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
// 只检查POST请求体
if ("POST".equalsIgnoreCase(request.getMethod())) {
ContentCachingRequestWrapper wrappedRequest =
new ContentCachingRequestWrapper(request);
chain.doFilter(wrappedRequest, response);
// 检查请求体内容
String body = wrappedRequest.getContentAsString();
for (Pattern pattern : INJECTION_PATTERNS) {
if (pattern.matcher(body).find()) {
response.setStatus(403);
response.getWriter().write(
"{\"error\":\"检测到安全风险\"}");
log.warn("Prompt注入尝试: body={}", body.substring(0, 100));
return;
}
}
} else {
chain.doFilter(request, response);
}
}
}
}40.9 监控与运维
40.9.1 自定义健康指标
java
@Component
public class AiHealthIndicator implements HealthIndicator {
private final ChatClient chatClient;
private final VectorStore vectorStore;
private final RedisTemplate<String, String> redisTemplate;
@Override
public Health health() {
Health.Builder builder = Health.up();
Map<String, Object> details = new HashMap<>();
// 1. 检查模型可用性
try {
String test = chatClient.prompt()
.user("ping")
.call()
.content();
details.put("chatModel", "UP");
details.put("chatModelResponse", test != null ? "ok" : "empty");
} catch (Exception e) {
builder = Health.down();
details.put("chatModel", "DOWN");
details.put("chatModelError", e.getMessage());
}
// 2. 检查向量库
try {
vectorStore.similaritySearch(SearchRequest.query("test").topK(1));
details.put("vectorStore", "UP");
} catch (Exception e) {
builder = Health.down();
details.put("vectorStore", "DOWN");
}
// 3. 检查Redis
try {
redisTemplate.opsForValue().get("health:check");
details.put("redis", "UP");
} catch (Exception e) {
builder = Health.down();
details.put("redis", "DOWN");
}
// 4. 检查今日Token用量
String today = LocalDate.now().toString();
String usage = redisTemplate.opsForValue().get("ai:tokens:" + today);
details.put("todayTokenUsage", usage != null ? usage : "0");
return builder.withDetails(details).build();
}
}40.9.2 自定义指标
java
@Component
@RequiredArgsConstructor
public class AiMetrics {
private final MeterRegistry meterRegistry;
/**
* 记录AI请求
*/
public void recordRequest(String model, boolean cacheHit, boolean success) {
meterRegistry.counter("ai.requests.total",
"model", model,
"cache_hit", String.valueOf(cacheHit),
"status", success ? "success" : "error"
).increment();
}
/**
* 记录Token消耗
*/
public void recordTokens(String model, int inputTokens, int outputTokens) {
meterRegistry.counter("ai.tokens.input", "model", model).increment(inputTokens);
meterRegistry.counter("ai.tokens.output", "model", model).increment(outputTokens);
}
/**
* 记录响应延迟
*/
public void recordLatency(String model, long durationMs, boolean stream) {
meterRegistry.timer("ai.response.duration",
"model", model,
"stream", String.valueOf(stream)
).record(Duration.ofMillis(durationMs));
}
/**
* 记录RAG检索
*/
public void recordRagRetrieval(int retrievedCount, long retrievalMs) {
meterRegistry.gauge("ai.rag.retrieved.count", retrievedCount);
meterRegistry.timer("ai.rag.retrieval.duration")
.record(Duration.ofMillis(retrievalMs));
}
/**
* 记录幻觉检测结果
*/
public void recordHallucinationCheck(boolean passed) {
meterRegistry.counter("ai.hallucination.check",
"result", passed ? "pass" : "fail"
).increment();
}
/**
* 更新预算使用率
*/
public void updateBudgetRatio(double ratio) {
meterRegistry.gauge("ai.budget.ratio", ratio);
}
}40.9.3 Grafana 监控面板
配置 application.yml 暴露Prometheus指标后，Grafana面板关键指标：
| 面板 | 指标 | PromQL |
|---|---|---|
| QPS | 每秒请求数 | rate(ai_requests_total[1m]) |
| 延迟P99 | 99分位响应时间 | histogram_quantile(0.99, rate(ai_response_duration_bucket[5m])) |
| Token消耗 | 每分钟Token | rate(ai_tokens_input_total[1m]) + rate(ai_tokens_output_total[1m]) |
| 缓存命中率 | 命中率 | rate(ai_requests_total{cache_hit="true"}[5m]) / rate(ai_requests_total[5m]) |
| 错误率 | 错误请求占比 | rate(ai_requests_total{status="error"}[5m]) / rate(ai_requests_total[5m]) |
| 预算使用 | 预算消耗率 | ai_budget_ratio |
| RAG检索数 | 平均检索数 | avg(ai_rag_retrieved_count) |
| 幻觉率 | 幻觉失败率 | rate(ai_hallucination_check_total{result="fail"}[1h]) |
40.10 评估与迭代
40.10.1 离线评估
java
@Service
public class EvaluationService {
private final ChatClient chatClient;
private final RagService ragService;
/**
* 离线评估 - 用测试集评估RAG质量
*/
public EvaluationReport evaluateRag(List<TestCase> testCases) {
List<TestResult> results = new ArrayList<>();
for (TestCase testCase : testCases) {
long startTime = System.currentTimeMillis();
// 执行RAG查询
RagResponse response = ragService.query(
"eval-user", "eval-session", testCase.question()
);
long duration = System.currentTimeMillis() - startTime;
// LLM-as-Judge 评估
EvaluationResult evalResult = evaluateWithLLM(
testCase.question(),
response.answer(),
testCase.expectedAnswer(),
testCase.context()
);
results.add(TestResult.builder()
.question(testCase.question())
.answer(response.answer())
.expectedAnswer(testCase.expectedAnswer())
.accuracy(evalResult.accuracy())
.faithfulness(evalResult.faithfulness())
.completeness(evalResult.completeness())
.relevance(evalResult.relevance())
.languageQuality(evalResult.languageQuality())
.overallScore(evalResult.overallScore())
.durationMs(duration)
.retrievedCount(response.retrievedCount())
.hallucinationWarning(response.hallucinationWarning())
.build());
}
// 生成汇总报告
return generateReport(results);
}
/**
* LLM-as-Judge
*/
private EvaluationResult evaluateWithLLM(String question, String answer,
String expected, String context) {
String evalPrompt = """
请评估以下AI回答的质量，从5个维度评分（1-5分）：
【问题】%s
【AI回答】%s
【参考答案】%s
【检索上下文】%s
评分维度：
1. 准确性：回答是否正确，与参考答案是否一致
2. 忠实度：回答是否基于上下文，有无幻觉
3. 完整性：回答是否覆盖了问题的所有要点
4. 相关性：回答是否与问题紧密相关
5. 语言质量：表达是否清晰、流畅、有条理
请输出JSON格式：
{"accuracy":5, "faithfulness":5, "completeness":4,
"relevance":5, "languageQuality":4, "overallScore":4.6,
"reason": "评价理由"}
""".formatted(question, answer, expected, context);
String result = chatClient.prompt()
.user(evalPrompt)
.call()
.content();
return parseEvaluationResult(result);
}
private EvaluationReport generateReport(List<TestResult> results) {
double avgAccuracy = results.stream()
.mapToDouble(TestResult::accuracy).average().orElse(0);
double avgFaithfulness = results.stream()
.mapToDouble(TestResult::faithfulness).average().orElse(0);
double avgCompleteness = results.stream()
.mapToDouble(TestResult::completeness).average().orElse(0);
double avgRelevance = results.stream()
.mapToDouble(TestResult::relevance).average().orElse(0);
double avgLanguage = results.stream()
.mapToDouble(TestResult::languageQuality).average().orElse(0);
double avgDuration = results.stream()
.mapToDouble(TestResult::durationMs).average().orElse(0);
long hallucinationCount = results.stream()
.filter(TestResult::hallucinationWarning).count();
return EvaluationReport.builder()
.totalCases(results.size())
.avgAccuracy(avgAccuracy)
.avgFaithfulness(avgFaithfulness)
.avgCompleteness(avgCompleteness)
.avgRelevance(avgRelevance)
.avgLanguageQuality(avgLanguage)
.avgOverallScore((avgAccuracy + avgFaithfulness + avgCompleteness
+ avgRelevance + avgLanguage) / 5)
.avgDurationMs(avgDuration)
.hallucinationRate((double) hallucinationCount / results.size())
.results(results)
.timestamp(LocalDateTime.now())
.build();
}
}40.10.2 Bad Case 管理
java
@Service
public class BadCaseService {
private final BadCaseRepository repository;
private final EvaluationService evaluationService;
/**
* 用户反馈 - 差评自动加入Bad Case
*/
@EventListener
public void handleNegativeFeedback(UserFeedbackEvent event) {
if (event.rating() <= 2) { // 1-2星为差评
BadCase badCase = BadCase.builder()
.question(event.question())
.answer(event.answer())
.expectedAnswer("") // 待人工标注
.feedback(event.feedback())
.rating(event.rating())
.userId(event.userId())
.timestamp(LocalDateTime.now())
.status(BadCaseStatus.PENDING_REVIEW)
.build();
repository.save(badCase);
log.info("差评自动加入Bad Case库: question={}",
event.question().substring(0, Math.min(50, event.question().length())));
}
}
/**
* 批量审核Bad Case
*/
@Scheduled(cron = "0 0 10 * * MON") // 每周一上午10点
public void batchReviewBadCases() {
List<BadCase> pending = repository.findByStatus(BadCaseStatus.PENDING_REVIEW);
log.info("本周待审核Bad Case: {}条", pending.size());
for (BadCase badCase : pending) {
// 自动生成参考答案
String reference = generateReferenceAnswer(badCase.getQuestion());
badCase.setExpectedAnswer(reference);
badCase.setStatus(BadCaseStatus.AUTO_REVIEWED);
repository.save(badCase);
}
}
/**
* 从Bad Case生成测试用例
*/
public List<TestCase> convertToTestCases() {
return repository.findByStatus(BadCaseStatus.VERIFIED).stream()
.map(bc -> new TestCase(bc.getQuestion(), bc.getExpectedAnswer(),
bc.getCategory()))
.collect(Collectors.toList());
}
}40.11 部署架构
40.11.1 Docker Compose
yaml
# docker-compose.yml
version: '3.8'
services:
ai-app:
build: .
ports:
- "8080:8080"
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}
- SPRING_PROFILES_ACTIVE=prod
- DB_HOST=postgres
- REDIS_HOST=redis
depends_on:
- postgres
- redis
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/actuator/health"]
interval: 30s
timeout: 10s
retries: 3
postgres:
image: pgvector/pgvector:pg16
environment:
POSTGRES_DB: ai_app
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
restart: unless-stopped
redis:
image: redis:7-alpine
ports:
- "6379:6379"
command: redis-server --maxmemory 512mb --maxmemory-policy allkeys-lru
restart: unless-stopped
volumes:
pgdata:40.11.2 Kubernetes 部署
yaml
# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-app
spec:
replicas: 3
selector:
matchLabels:
app: ai-app
template:
spec:
containers:
- name: ai-app
image: registry.example.com/ai-app:1.0.0
ports:
- containerPort: 8080
env:
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: ai-secrets
key: openai-api-key
resources:
requests:
memory: "1Gi"
cpu: "500m"
limits:
memory: "2Gi"
cpu: "1000m"
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 60
periodSeconds: 30
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: ai-app-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ai-app
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 8040.12 全书总结
40.12.1 7个阶段40章全景回顾
| 阶段 | 章节 | 核心知识 | 大白话 |
|---|---|---|---|
| 第1阶段 认知启蒙 | 第1-4章 | AI简史、AI/ML/DL关系、能力边界、三条路线 | 先知道AI是啥，再谈怎么用 |
| 第2阶段 机器学习基础 | 第5-9章 | 监督/无监督学习、特征工程、模型评估、实操 | 让机器学会看数据找规律 |
| 第3阶段 深度学习 | 第10-14章 | 神经网络、CNN、RNN、Transformer、框架实操 | 神经网络到底怎么"学习"的 |
| 第4阶段 大模型（LLM） | 第15-20章 | LLM原理、模型选型、Prompt、微调、量化部署、评估 | GPT们到底是怎么工作的 |
| 第5阶段 RAG检索增强生成 | 第21-27章 | RAG原理、数据处理、向量化、向量库、检索策略、生成优化、框架 | 给AI装上外挂知识库 |
| 第6阶段 Agent智能体 | 第28-33章 | Agent概念、Function Call、ReAct、工作流、记忆、实战 | 让AI学会用工具干活 |
| 第7阶段 AI工程化落地 | 第34-40章 | 架构设计、成本控制、性能优化、安全防护、可观测性、评估迭代、Spring AI实战 | 从Demo到生产，差的不止一点 |
40.12.2 核心知识图谱
AI开发知识全景
│
├─ 认知层（第1阶段）
│ └─ 理解AI能干什么、不能干什么 → 决定用什么路线
│
├─ 基础层（第2-3阶段）
│ ├─ 机器学习：传统ML在表格数据上仍然不可替代
│ └─ 深度学习：CNN看图、RNN看序列、Transformer看一切
│ └─ Transformer → 所有现代大模型的架构基础
│
├─ 应用层（第4-6阶段）
│ ├─ LLM：Next Token Prediction → 千亿参数 → 智能涌现
│ │ ├─ 使用：Prompt工程（零成本优化，先做这个）
│ │ ├─ 定制：微调（LoRA/QLoRA，改能力不改知识）
│ │ ├─ 部署：量化（INT4）+ 推理引擎（vLLM）
│ │ └─ 评估：自有测试集 > 通用Benchmark
│ │
│ ├─ RAG：检索→增强→生成（给模型开卷考试）
│ │ ├─ 数据处理 → 切分策略决定检索质量上限
│ │ ├─ 向量化 → BGE选型，成本极低
│ │ ├─ 向量库 → PgVector够用，简单优先
│ │ ├─ 检索策略 → 改写+混合(RRF)+重排序三板斧
│ │ └─ 生成优化 → 上下文组装+引用标注+幻觉检测
│ │
│ └─ Agent：自主决策+工具使用
│ ├─ Function Call → 模型发令，系统执行
│ ├─ ReAct → Think-Act-Observe循环
│ ├─ 工作流 → 多Agent协作（LangGraph）
│ └─ 记忆 → 短期(滑动窗口)+长期(向量/结构化)
│
└─ 工程层（第7阶段）
├─ 架构设计 → AI网关+模型路由+三级缓存+五级降级
├─ 成本控制 → 缓存+路由+Prompt精简 → 降50%+
├─ 性能优化 → 流式输出+KV Cache+量化 → 体验关键
├─ 安全防护 → 四层防御+Function Call安全+多租户隔离
├─ 可观测性 → 指标+链路+日志+质量监控(第四支柱)
├─ 评估迭代 → 闭环：监控→分析→优化→评估→上线
└─ Spring AI → Java团队的企业级AI开发框架40.12.3 给开发者的建议
如果你是后端开发者想转AI开发：
- 第1-3阶段快速过 --- 理解概念，不死磕数学。重点理解Transformer
- 第4阶段重点投入 --- LLM是当前AI应用的核心。Prompt工程是基本功
- 第5阶段必须掌握 --- RAG是后端开发者最容易上手的AI应用场景
- 第6阶段按需学习 --- Agent是趋势但成熟度不够，别过度投入
- 第7阶段是你的主场 --- 后端工程能力是AI落地的核心竞争力
技术选型建议：
| 场景 | 推荐方案 | 理由 |
|---|---|---|
| 快速验证AI效果 | API + Prompt + RAG | 零成本起步，先验证再投入 |
| 企业知识库 | PgVector + BGE + LangChain/Spring AI | 简单够用，运维成熟 |
| 智能客服 | Spring AI + Function Call + RAG | Java团队首选，企业级生态完善 |
| 高并发AI服务 | vLLM + Redis缓存 + 模型路由 | 量化+缓存+路由三板斧 |
| 复杂Agent系统 | LangGraph + 多Agent协作 | 可控性强，适合生产 |
| 个人知识助手 | Ollama + 本地模型 | 隐私+免费+够用 |
最重要的三句话：
- 先API再自部署 --- 90%的场景API就够了，别一开始就折腾模型部署
- 先Prompt再微调 --- Prompt没优化好别提微调，微调改能力不改知识
- 先简单再复杂 --- 能用单Agent解决别上多Agent，能用PgVector别上Milvus，能用规则解决别上AI