文章目录
一、前言
Spring AI实战初体验------实现可切换模型AI聊天助手-CSDN博客
如上，在上一篇博客，我们已经实现了spring ai对接本地大模型实现了聊天机器人，但是目前有个新需求：
- 上传某场所的图片，通过AI进行分析，描述图片里的内容以及存在的安全隐患
- 进一步通过AI分析场所的安全隐患如何治理，需要依据法律法规（联网）分析
最终效果如下所示：
由于目前了解到的本地大模型都无法实现上述的需求，于是这次借助了火山引擎平台来实现
https://console.volcengine.com/ark/
火山引擎目前新用户会赠送每个模型50万token的体验量，对于学习、测试用还是足够的
如下所示，本次对接的模型有 doubao-vision-pro（图片识别）和 deepseek-v3（联网分析）
整体的逻辑：
- 先传入图片到doubao模型，分析图片里的场所和存在的隐患
- 然后将1分析的文字结果传到deepseek-v3模型联网结合法律法规分析隐患的整改措施
二、创建应用
https://console.volcengine.com/ark/
如下所示，创建2个零代码应用
-
图片识别
-
联网分析
三、后端
1.SDK集成
如下图所示，火山引擎里部分模型像deepseek-v3是可以直接集成SDK来对接的
代码示例
java
@RestController
@RequestMapping("/huoShan")
public class HuoShanController {
private final ArkService service;
private final String imageAnalyzeBotId;
public HuoShanController(@Value("${ai.ark.apiKey}") String apiKey, @Value("${ai.ark.base-url}") String baseUrl, @Value("${ai.ark.image-analyze-botId}") String imageAnalyzeBotId) {
this.imageAnalyzeBotId = imageAnalyzeBotId;
this.service = ArkService.builder()
.dispatcher(new Dispatcher())
.connectionPool(new ConnectionPool(5, 1, TimeUnit.SECONDS))
.baseUrl(baseUrl)
.apiKey(apiKey)
.build();
}
@PostMapping("/image/chat")
public ResponseEntity<String> imageChat(@RequestBody String userMessage) {
List<ChatMessage> messages = new ArrayList<>();
messages.add(ChatMessage.builder().role(ChatMessageRole.SYSTEM).content("你是一个对中国法律法规有深入理解的专家").build());
messages.add(ChatMessage.builder().role(ChatMessageRole.USER).content(userMessage).build());
BotChatCompletionRequest chatCompletionRequest = BotChatCompletionRequest.builder()
.botId(imageAnalyzeBotId)
.messages(messages)
.build();
BotChatCompletionResult chatCompletionResult = service.createBotChatCompletion(chatCompletionRequest);
StringBuilder result = new StringBuilder();
chatCompletionResult.getChoices().forEach(choice -> result.append(choice.getMessage().getContent()));
return ResponseEntity.ok(result.toString());
}
}相关的apiKey、base-url、botId都可以从火山的API调用指南获取，获取完我们配置在application.yml里就可以从上面的代码获取
2.调用Rest API
有些模型例如doubao-pro-vision就没提供java SDK，所以需要采用直接调用rest api的方式来对接
代码示例
java
@PostMapping("/notStream")
public Mono<String> imageAnalysis(MultipartFile file) {
return imageAiService.imageAnalysisNotStream(file);
}
java
@Service
@Slf4j
public class ImageAiService {
@Value("${ai.ark.image-botId}")
private String MODEL;
private final WebClient webClient;
public ImageAiService(@Value("${ai.ark.apiKey}") String apiKey, @Value("${ai.ark.base-url}") String baseUrl) {
this.webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()))
.baseUrl(baseUrl)
.defaultHeader("Authorization", "Bearer "+ apiKey)
.build();
public Mono<String> imageAnalysisNotStream(MultipartFile file) {
// 压缩图片并转成 Base64 格式
String base64 = ImageCompressor.compressImageFileToBase64UnderSize(file, 400, 400, 100);
if (base64 == null || base64.isEmpty()) {
log.error("图片压缩失败");
return Mono.just("图片压缩失败");
}
// 构造请求体
Map<String, Object> body = new HashMap<>();
body.put("model", MODEL);
// 非流式返回
body.put("stream", false);
body.put("stream_options", Map.of("include_usage", true));
Map<String, Object> imageContent = Map.of(
"type", "image_url",
"image_url", Map.of("url", base64)
);
Map<String, Object> message = Map.of(
"role", "user",
"content", List.of(imageContent)
);
body.put("messages", List.of(message));
// 调用非流式接口，直接返回拼接后的完整结果字符串
return webClient.post()
.uri("/bots/chat/completions")
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(body)
// 此时接口返回的是 JSON 数据，所以指定 JSON 类型
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(String.class)
.map(responseStr -> {
try {
// 解析返回结果，取出 assistant 返回的内容
JsonNode jsonNode = new ObjectMapper().readTree(responseStr);
// 此处根据实际返回结构调整解析逻辑
JsonNode contentNode = jsonNode.path("choices")
.get(0)
.path("message")
.path("content");
return contentNode.asText();
} catch (Exception e) {
log.error("解析返回结果异常", e);
return "解析返回结果异常";
}
});
}
}注意：
上述调用火山引擎api都是非流式的，如果流式输出就把stream设置成true，再使用Flux类或SseEmitter类去接收返回就行，但是由于我流式输出得到的结果前端进行格式处理时候总是有问题，所以改用了非流式，等完整答案出来后再一次性处理格式化
java
/**
* 压缩到不超过100KB的Base64编码
*/
public static String compressImageFileToBase64UnderSize(MultipartFile file, int maxWidth, int maxHeight, int maxSizeKB) {
try {
// 读取 MultipartFile 图片
BufferedImage originalImage = ImageIO.read(file.getInputStream());
// 按比例缩放图片
Image scaledImage = originalImage.getScaledInstance(maxWidth, maxHeight, Image.SCALE_SMOOTH);
BufferedImage resizedImage = new BufferedImage(maxWidth, maxHeight, BufferedImage.TYPE_INT_RGB);
Graphics2D g2d = resizedImage.createGraphics();
g2d.drawImage(scaledImage, 0, 0, null);
g2d.dispose();
// 获取JPEG写入器
Iterator<ImageWriter> writers = ImageIO.getImageWritersByFormatName("jpg");
if (!writers.hasNext()) throw new IllegalStateException("No writers found for jpg");
ImageWriter writer = writers.next();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
MemoryCacheImageOutputStream output = new MemoryCacheImageOutputStream(baos);
writer.setOutput(output);
// 设置初始压缩质量
float quality = 1.0f;
byte[] imageBytes;
do {
baos.reset();
ImageWriteParam param = writer.getDefaultWriteParam();
param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
param.setCompressionQuality(quality);
writer.write(null, new IIOImage(resizedImage, null, null), param);
output.flush();
imageBytes = baos.toByteArray();
quality -= 0.05f; // 每次降低压缩质量
} while (imageBytes.length > maxSizeKB * 1024 && quality > 0.05f);
writer.dispose();
output.close();
//System.out.println("Final image size: " + (imageBytes.length / 1024) + " KB, final quality: " + quality);
return "data:image/jpeg;base64," + Base64.getEncoder().encodeToString(imageBytes);
} catch (IOException e) {
e.printStackTrace();
return null;
}
}注意：
java
Map<String, Object> imageContent = Map.of(
"type", "image_url",
"image_url", Map.of("url", base64)
);这里的图片可以传递http/https网络地址 或者图片的base64编码，由于我想是用电脑本地的文件来测试，所以采用图片转base64编码的方式来传递
四、前端
基于上次的html，新增了图片上传，图片回显，调用新接口等处理
直接放完整代码：
html
<!DOCTYPE html>
<html lang="zh">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI 聊天</title>
<style>
html, body {
height: 100%;
width: 100%;
margin: 0;
background-color: #f9f9f9;
display: flex;
align-items: center;
justify-content: center;
}
.container {
display: flex;
flex-direction: column;
height: 90vh;
max-width: 800px;
width: 100%;
margin: auto;
}
.chat-container {
flex: 1;
display: flex;
flex-direction: column;
background: white;
border-radius: 10px;
padding: 20px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
overflow-y: auto; /* 确保内容超出时显示滚动条 */
min-height: 0; /* 防止 flex 容器压缩子元素 */
}
.chat-container::-webkit-scrollbar {
width: 8px;
}
.chat-container::-webkit-scrollbar-track {
background: #f1f1f1;
border-radius: 4px;
}
.chat-container::-webkit-scrollbar-thumb {
background: #888;
border-radius: 4px;
}
.chat-container::-webkit-scrollbar-thumb:hover {
background: #555;
}
.ai-message h3 {
font-size: 1.2em;
margin-top: 1em;
}
.ai-message ul {
padding-left: 1.5em;
}
.ai-message li {
margin-bottom: 0.5em;
}
.ai-message {
white-space: pre-wrap;
}
.loading-spinner {
border: 4px solid #f3f3f3;
border-top: 4px solid #007bff;
border-radius: 50%;
width: 30px;
height: 30px;
animation: spin 1s linear infinite;
margin: 10px auto;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
.message {
padding: 10px 15px;
border-radius: 15px;
margin: 5px 0;
max-width: 80%;
word-wrap: break-word;
}
.user-message {
background-color: #007bff;
color: white;
align-self: flex-end;
}
.ai-message {
background-color: #e5e5e5;
color: black;
align-self: flex-start;
}
.think-message {
background-color: #add8e6;
color: black;
border-radius: 10px;
padding: 10px;
margin: 5px 0;
max-width: 80%;
align-self: flex-start;
font-style: italic;
}
.think-content {
flex: 1; /* 允许内容自由扩展 */
overflow-y: auto; /* 内容过多时显示滚动条 */
padding: 5px;
}
.think-title {
font-weight: bold;
margin-bottom: 5px;
display: flex;
align-items: center;
}
.toggle-button {
padding: 5px 10px;
background-color: #007bff;
color: white;
border: none;
border-radius: 5px;
cursor: pointer;
margin-right: 10px;
}
.toggle-button:hover {
background-color: #0056b3;
}
.input-container {
display: flex;
flex-direction: column;
padding: 10px;
background: white;
box-shadow: 0 -2px 5px rgba(0, 0, 0, 0.1);
}
.model-container {
display: flex;
align-items: center;
margin-bottom: 5px;
}
.model-label {
margin-right: 10px;
font-weight: bold;
}
.model-select {
padding: 5px;
border-radius: 5px;
border: 1px solid #ccc;
}
.input-box-container {
display: flex;
align-items: center;
}
.input-box {
flex: 1;
padding: 10px;
border: 1px solid #ccc;
border-radius: 5px;
}
.send-button, .clear-button, .stop-button {
padding: 10px 20px;
margin-left: 10px;
color: white;
border: none;
border-radius: 5px;
cursor: pointer;
}
.send-button { background-color: #007bff; }
.send-button:hover { background-color: #0056b3; }
.send-button:disabled { background-color: #a0c4ff; cursor: not-allowed; }
.clear-button { background-color: #dc3545; }
.clear-button:hover { background-color: #a71d2a; }
.clear-button:disabled { background-color: #f5a6a6; cursor: not-allowed; }
.stop-button { background-color: #ff9800; }
.stop-button:hover { background-color: #e68900; }
.stop-button:disabled { background-color: #ffb74d; cursor: not-allowed; }
</style>
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
</head>
<body>
<div class="container">
<div class="chat-container" id="chatContainer">
<div class="message ai-message">👋 你好，我是你的 AI 助手！</div>
</div>
<div class="input-container">
<div class="model-container">
<span class="model-label">选择模型:</span>
<select id="modelSelect" class="model-select" onchange="changeModel()">
<option value="deepseek-r1:latest">DeepSeek-R1（推理）</option>
<option value="qwen:7b">Qwen</option>
<option value="image-analysis">火山引擎-Doubao（场所图片分析，无记忆）</option>
</select>
</div>
<div class="input-box-container">
<input id="userInput" class="input-box" placeholder="请输入消息...">
<input id="imageUpload" type="file" accept="image/jpeg, image/png" style="display: none;" onchange="validateFile()" />
<button id="sendButton" class="send-button" onclick="handleSend()">发送</button>
<button id="clearButton" class="clear-button" onclick="clearMemory()">清除上下文</button>
<button id="stopButton" class="stop-button" onclick="stopAIResponse()">停止回答</button>
</div>
</div>
</div>
<script>
const chatContainer = document.getElementById('chatContainer');
const userInput = document.getElementById('userInput');
const modelSelect = document.getElementById('modelSelect');
const sendButton = document.getElementById('sendButton');
const clearButton = document.getElementById('clearButton');
const stopButton = document.getElementById('stopButton');
let userId = '1';
let currentModel = modelSelect.value;
let eventSource = null;
function validateFile() {
const fileInput = document.getElementById('imageUpload');
const file = fileInput.files[0];
if (file) {
const fileSize = file.size / 1024; // 文件大小，单位为KB
const fileType = file.type.toLowerCase();
// 判断文件大小是否小于200KB，格式是否为JPG或PNG
if (fileSize > 200) {
alert('文件大小必须小于200KB。');
fileInput.value = ''; // 清空选择框
return;
}
if (fileType !== 'image/jpeg' && fileType !== 'image/png') {
alert('仅允许上传JPG和PNG格式的图片。');
fileInput.value = ''; // 清空选择框
return;
}
}
}
function handleSend() {
if (currentModel === 'image-analysis') {
const fileInput = document.getElementById('imageUpload');
const file = fileInput.files[0];
if (!file) {
alert("请选择图片文件");
return;
}
uploadAndAnalyzeImage(file);
} else {
sendMessage();
}
}
function uploadAndAnalyzeImage(file) {
const formData = new FormData();
formData.append('file', file);
// 回显图片
const reader = new FileReader();
reader.onload = function(e) {
const imgElement = document.createElement('img');
imgElement.src = e.target.result;
imgElement.style.maxWidth = '200px';
imgElement.style.borderRadius = '10px';
imgElement.style.margin = '10px 0';
const userImgMessage = document.createElement('div');
userImgMessage.classList.add('message', 'user-message');
userImgMessage.appendChild(imgElement);
chatContainer.appendChild(userImgMessage);
toggleAllButtons(false); // 禁用按钮
showLoadingSpinner(); // 显示加载动画
chatContainer.scrollTop = chatContainer.scrollHeight;
};
reader.readAsDataURL(file);
fetch('http://192.168.100.72:8081/image/notStream', {
method: 'POST',
body: formData
})
.then(response => response.text())
.then(text => {
const fixedText = text.replace(/\\n/g, '\n');
const html = marked.parse(fixedText);
const aiMessage = document.createElement('div');
aiMessage.classList.add('message', 'ai-message');
aiMessage.innerHTML = html;
chatContainer.appendChild(aiMessage);
// 添加进一步分析提示与按钮
const followUp = document.createElement('div');
followUp.classList.add('message', 'ai-message');
followUp.innerHTML = `
<div style="display: flex; align-items: center;">
<span style="margin-right: 10px;">是否进一步分析风险隐患及对应整改措施？</span>
<button class="send-button" onclick="startRiskAnalysis(\`${text.replace(/`/g, '\\`')}\`)">是</button>
</div>
`;
chatContainer.appendChild(followUp);
})
.catch(err => {
console.error("图片分析请求失败:", err);
appendMessage("❌ 图片分析失败", "ai-message");
})
.finally(() => {
hideLoadingSpinner(); // 移除加载动画
toggleAllButtons(true); // 启用按钮
chatContainer.scrollTop = chatContainer.scrollHeight;
});
}
function startRiskAnalysis(content) {
toggleAllButtons(false); // 禁用按钮
// 显示转圈动画
showLoadingSpinner();
fetch('http://192.168.100.72:8081/huoShan/image/chat', {
method: 'POST',
headers: {
'Content-Type': 'text/plain'
},
body: content
})
.then(response => response.text()) // 获取文本响应
.then(result => {
// 隐藏转圈动画
hideLoadingSpinner()
// 格式化结果并解析为 HTML
const fixedText = result.replace(/\\n/g, '\n'); // 解除转义
const html = marked.parse(fixedText);
// 创建新的 AI 消息并添加到界面
const aiMessage = document.createElement('div');
aiMessage.classList.add('message', 'ai-message');
aiMessage.innerHTML = html;
chatContainer.appendChild(aiMessage);
chatContainer.scrollTop = chatContainer.scrollHeight; // 滚动到底部
toggleAllButtons(true); // 启用按钮
})
.catch(err => {
console.error("风险分析请求失败:", err);
// 隐藏转圈动画并显示失败消息
hideLoadingSpinner()
appendMessage("❌ 风险分析请求失败", "ai-message");
toggleAllButtons(true); // 启用按钮
});
}
function showLoadingSpinner() {
const spinner = document.createElement('div');
spinner.id = 'loadingSpinner';
spinner.className = 'loading-spinner';
chatContainer.appendChild(spinner);
chatContainer.scrollTop = chatContainer.scrollHeight;
}
function hideLoadingSpinner() {
const spinner = document.getElementById('loadingSpinner');
if (spinner) spinner.remove();
}
function sendMessage() {
let message = userInput.value.trim();
if (!message) return;
appendMessage(message, 'user-message');
streamAIResponse(userId, message);
userInput.value = '';
}
function appendMessage(text, type) {
const messageElement = document.createElement('div');
messageElement.classList.add('message', type);
messageElement.textContent = text;
chatContainer.appendChild(messageElement);
chatContainer.scrollTop = chatContainer.scrollHeight;
}
function streamAIResponse(userId, message) {
// 先终止可能存在的旧 eventSource
if (eventSource) {
eventSource.close();
}
eventSource = new EventSource(`http://192.168.100.72:8081/ai/chatStreamWithMemory?userId=${encodeURIComponent(userId)}&message=${encodeURIComponent(message)}&model=${encodeURIComponent(currentModel)}`);
let aiMessage = null;
let thinkMode = false;
let thinkMessage = null;
eventSource.onmessage = event => {
let response = event.data;
if (response.includes('<think>') && currentModel === 'deepseek-r1:latest') {
thinkMode = true;
response = response.replace('<think>', '');
// 创建思考过程气泡
thinkMessage = document.createElement('div');
thinkMessage.classList.add('think-message');
thinkMessage.innerHTML = `
<div class="think-title">
<button class="toggle-button" onclick="toggleThinkMessage(this)">折叠</button>
<span class="think-title-text">思考过程：</span>
</div>
<div class="think-content" style="display: block;"></div>
`;
chatContainer.appendChild(thinkMessage);
}
if (thinkMode) {
const thinkContent = thinkMessage.querySelector('.think-content');
if (response.includes('</think>')) {
response = response.replace('</think>', '');
thinkMode = false;
aiMessage = document.createElement('div');
aiMessage.classList.add('message', 'ai-message');
chatContainer.appendChild(aiMessage);
}
thinkContent.innerHTML += response;
} else {
if (!aiMessage) {
aiMessage = document.createElement('div');
aiMessage.classList.add('message', 'ai-message');
chatContainer.appendChild(aiMessage);
}
aiMessage.textContent += response;
}
chatContainer.scrollTop = chatContainer.scrollHeight;
};
eventSource.onerror = () => {
eventSource.close();
toggleButtons(true);
};
eventSource.onopen = () => {
toggleButtons(false);
};
eventSource.addEventListener("close", () => {
toggleButtons(true);
});
}
function toggleAllButtons(enabled) {
sendButton.disabled = !enabled;
clearButton.disabled = !enabled;
stopButton.disabled = !enabled;
}
function toggleButtons(enabled) {
sendButton.disabled = !enabled;
clearButton.disabled = !enabled;
}
function toggleThinkMessage(button) {
const thinkMessage = button.closest('.think-message');
const thinkContent = thinkMessage.querySelector('.think-content');
if (thinkContent.style.display === 'none') {
thinkContent.style.display = 'block';
button.textContent = '折叠';
} else {
thinkContent.style.display = 'none';
button.textContent = '展开';
}
}
function stopAIResponse() {
if (eventSource) {
eventSource.close();
eventSource = null;
}
fetch(`http://192.168.100.72:8081/ai/stopChat?userId=${userId}`, { method: 'GET' })
.then(() => appendMessage('AI 回答已停止。', 'ai-message'))
.catch(error => console.error(error));
}
function clearMemory() {
fetch(`http://192.168.100.72:8081/ai/clearMemory?userId=${userId}`, { method: 'GET' })
.then(() => appendMessage('上下文已清除。', 'ai-message'))
.catch(error => console.error(error));
}
function changeModel() {
currentModel = modelSelect.value;
let currentModelName = modelSelect.options[modelSelect.selectedIndex].text;
appendMessage(`已切换模型为 ${currentModelName}`, 'ai-message');
if (currentModel === 'image-analysis') {
userInput.disabled = true;
userInput.placeholder = '请选择图片进行分析...';
sendButton.textContent = '分析图片';
document.getElementById('imageUpload').style.display = 'block';
} else {
userInput.disabled = false;
userInput.placeholder = '请输入消息...';
sendButton.textContent = '发送';
document.getElementById('imageUpload').style.display = 'none';
}
}
userInput.addEventListener('keypress', event => {
if (event.key === 'Enter') {
sendMessage();
}
});
</script>
</body>
</html>