✨道路是曲折的，前途是光明的！
📝 专注C/C++、Linux编程与人工智能领域，分享学习笔记！
🌟 感谢各位小伙伴的长期陪伴与支持，欢迎文末添加好友一起交流！
随着 AIGC 的爆发式发展，数字人技术正从科幻走向现实。本文将带你从零开始，构建一个基于 Web 的实时交互式数字人系统，涵盖数字人驱动、AI 对话、语音交互等核心技术。
技术背景
什么是数字人？
数字人（Digital Human）是指通过计算机图形学、人工智能等技术创建的、具有人类外观特征的虚拟角色。随着技术的发展，数字人已从简单的 3D 模型进化为能够实时交互、具备智能的"虚拟生命"。
应用场景
| 场景 | 描述 | 技术要点 |
|---|---|---|
| 虚拟主播 | 直播、新闻播报 | 实时驱动、表情同步 |
| 智能客服 | 企业服务、政务咨询 | 知识库、多轮对话 |
| 虚拟导师 | 在线教育、技能培训 | 教学交互、个性化 |
| 元宇宙社交 | 虚拟会议、社交游戏 | 多用户同步、沉浸感 |
技术栈选型
前端层: HTML5 + CSS3 + JavaScript
数字人SDK: 魔珐星云 / ReadyPlayerMe / MediaPipe
AI模型: OpenAI GPT / 阿里通义千问 / Claude
语音服务: Azure TTS / 讯飞 / Web Speech API
实时通信: WebRTC / WebSocket系统架构设计
整体架构图
数据层
服务层
应用层
用户端
浏览器界面
视频渲染区
对话交互区
数字人控制器
AI对话引擎
语音处理模块
状态管理器
数字人SDK
大语言模型API
TTS/ASR服务
知识库
配置存储
对话历史
用户画像
核心交互流程
语音服务 AI引擎 数字人控制器 界面 用户 语音服务 AI引擎 数字人控制器 界面 用户 输入问题/语音 触发交互 切换到listen状态 发送用户问题 构建Prompt 调用LLM 返回AI回复 切换到think状态 文字转语音 返回音频流 切换到speak状态 更新对话记录 切换到idle状态
状态机设计
数字人在交互过程中有多种状态，正确的状态管理是保证流畅体验的关键。
初始状态
用户点击连接
SDK初始化成功
初始化失败
默认待机
用户开始输入
长时间无交互
提交问题给AI
用户取消
AI返回回复
AI返回错误
播放完成
用户打断
用户唤醒
断开连接
Disconnected
Connecting
Connected
Idle
Listen
Offline
Think
Speak
待机状态
数字人播放待机动画
说话状态
驱动口型和表情
核心技术实现
1. 数字人SDK集成
SDK初始化
javascript
class AvatarController {
constructor(containerId, config) {
this.container = document.querySelector(containerId);
this.config = config;
this.sdk = null;
this.currentState = 'disconnected';
this.eventHandlers = new Map();
}
/**
* 初始化数字人SDK
*/
async init() {
try {
this.updateState('connecting');
// 创建SDK实例
this.sdk = new XmovAvatar({
containerId: this.container.id,
appId: this.config.appId,
appSecret: this.config.appSecret,
gatewayServer: 'https://nebula-agent.xingyun3d.com/user/v1/ttsa/session',
// 关键回调配置
onMessage: (msg) => this.handleMessage(msg),
onStateChange: (state) => this.handleStateChange(state),
onStatusChange: (status) => this.handleStatusChange(status),
onVoiceStateChange: (voiceState) => this.handleVoiceStateChange(voiceState),
});
await this.sdk.init();
this.updateState('connected');
this.idle(); // 进入待机状态
return true;
} catch (error) {
console.error('SDK初始化失败:', error);
this.updateState('disconnected');
throw error;
}
}
/**
* 状态管理
*/
updateState(newState) {
const oldState = this.currentState;
this.currentState = newState;
this.emit('stateChange', { oldState, newState });
}
/**
* 待机模式
*/
idle() {
if (this.sdk) {
this.sdk.idle();
this.updateState('idle');
}
}
/**
* 倾听模式
*/
listen() {
if (this.sdk) {
this.sdk.listen();
this.updateState('listen');
}
}
/**
* 思考模式
*/
think() {
if (this.sdk) {
this.sdk.think();
this.updateState('think');
}
}
/**
* 说话模式
*/
async speak(text) {
if (this.sdk) {
this.updateState('speak');
await this.sdk.speak(text);
this.updateState('idle');
}
}
/**
* 销毁实例
*/
destroy() {
if (this.sdk) {
this.sdk.destroy();
this.sdk = null;
this.updateState('disconnected');
}
}
// 事件处理方法
handleMessage(message) {
console.log('SDK消息:', message);
if (message.code !== 0) {
this.emit('error', message);
}
}
handleStateChange(state) {
console.log('状态变化:', state);
this.emit('avatarStateChange', state);
}
handleStatusChange(status) {
console.log('连接状态:', status);
this.emit('statusChange', status);
}
handleVoiceStateChange(voiceState) {
console.log('语音状态:', voiceState);
this.emit('voiceStateChange', voiceState);
}
// 事件系统
on(event, handler) {
if (!this.eventHandlers.has(event)) {
this.eventHandlers.set(event, []);
}
this.eventHandlers.get(event).push(handler);
}
emit(event, data) {
const handlers = this.eventHandlers.get(event) || [];
handlers.forEach(handler => handler(data));
}
}2. AI对话引擎
流式对话实现
javascript
class AIConversationEngine {
constructor(config) {
this.apiKey = config.apiKey;
this.model = config.model || 'qwen-plus';
this.baseURL = config.baseURL || 'https://api.modelscope.cn/v1';
this.systemPrompt = config.systemPrompt || this.getDefaultPrompt();
this.conversationHistory = [];
}
/**
* 默认系统提示词
*/
getDefaultPrompt() {
return `你是一个智能助手，名叫"小政"。
你的职责是为用户提供专业的咨询服务。
服务准则：
1. 用简洁、友好、专业的语言回答
2. 不确定的信息诚实告知
3. 超出范围的问题礼貌拒绝并引导
4. 回答控制在200字以内`;
}
/**
* 发送消息并获取流式响应
*/
async chat(userMessage, onChunk, onComplete, onError) {
try {
// 添加用户消息到历史
this.conversationHistory.push({
role: 'user',
content: userMessage
});
// 构建请求消息
const messages = [
{ role: 'system', content: this.systemPrompt },
...this.getRecentHistory(10) // 保留最近10轮对话
];
// 发起流式请求
const response = await fetch(`${this.baseURL}/chat/completions`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${this.apiKey}`
},
body: JSON.stringify({
model: this.model,
messages: messages,
stream: true,
temperature: 0.7,
max_tokens: 2000
})
});
if (!response.ok) {
throw new Error(`API请求失败: ${response.status}`);
}
// 处理流式响应
const reader = response.body.getReader();
const decoder = new TextDecoder();
let fullResponse = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
const lines = chunk.split('\n').filter(line => line.trim());
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = line.slice(6);
if (data === '[DONE]') continue;
try {
const parsed = JSON.parse(data);
const content = parsed.choices[0]?.delta?.content;
if (content) {
fullResponse += content;
onChunk(content); // 回调处理每个文本块
}
} catch (e) {
console.warn('解析chunk失败:', e);
}
}
}
}
// 保存助手回复到历史
this.conversationHistory.push({
role: 'assistant',
content: fullResponse
});
onComplete(fullResponse);
} catch (error) {
console.error('AI对话失败:', error);
onError(error);
}
}
/**
* 获取最近的对话历史
*/
getRecentHistory(limit) {
return this.conversationHistory.slice(-limit);
}
/**
* 清空对话历史
*/
clearHistory() {
this.conversationHistory = [];
}
/**
* 更新系统提示词
*/
updateSystemPrompt(newPrompt) {
this.systemPrompt = newPrompt;
}
}3. 语音处理模块
javascript
class VoiceProcessor {
constructor(config) {
this.config = config;
this.synthesis = window.speechSynthesis;
this.recognition = null;
this.isListening = false;
}
/**
* 文字转语音
*/
async speak(text, options = {}) {
return new Promise((resolve, reject) => {
// 取消之前的播放
this.synthesis.cancel();
const utterance = new SpeechSynthesisUtterance(text);
utterance.lang = options.lang || 'zh-CN';
utterance.rate = options.rate || 1.0;
utterance.pitch = options.pitch || 1.0;
utterance.volume = options.volume || 1.0;
// 选择语音包
if (options.voiceName) {
const voices = this.synthesis.getVoices();
const voice = voices.find(v => v.name === options.voiceName);
if (voice) utterance.voice = voice;
}
utterance.onend = () => resolve();
utterance.onerror = (error) => reject(error);
this.synthesis.speak(utterance);
});
}
/**
* 语音识别（需要用户授权）
*/
startListening(onResult, onError) {
if (!('webkitSpeechRecognition' in window)) {
onError(new Error('浏览器不支持语音识别'));
return;
}
this.recognition = new webkitSpeechRecognition();
this.recognition.lang = 'zh-CN';
this.recognition.continuous = false;
this.recognition.interimResults = true;
let finalTranscript = '';
this.recognition.onresult = (event) => {
let interimTranscript = '';
for (let i = event.resultIndex; i < event.results.length; i++) {
const transcript = event.results[i][0].transcript;
if (event.results[i].isFinal) {
finalTranscript += transcript;
} else {
interimTranscript += transcript;
}
}
onResult({
final: finalTranscript,
interim: interimTranscript
});
};
this.recognition.onerror = (event) => {
onError(new Error(event.error));
};
this.recognition.onend = () => {
this.isListening = false;
};
this.recognition.start();
this.isListening = true;
}
/**
* 停止语音识别
*/
stopListening() {
if (this.recognition && this.isListening) {
this.recognition.stop();
this.isListening = false;
}
}
/**
* 停止语音播放
*/
stopSpeaking() {
this.synthesis.cancel();
}
/**
* 获取可用的语音列表
*/
getVoices() {
return this.synthesis.getVoices();
}
}4. 主控制器集成
javascript
class DigitalHumanSystem {
constructor(config) {
this.config = config;
// 初始化各模块
this.avatar = new AvatarController('#avatar-container', {
appId: config.avatarAppId,
appSecret: config.avatarAppSecret
});
this.ai = new AIConversationEngine({
apiKey: config.aiApiKey,
model: config.aiModel,
systemPrompt: config.systemPrompt
});
this.voice = new VoiceProcessor(config.voice);
// UI状态
this.isProcessing = false;
}
/**
* 初始化系统
*/
async init() {
// 初始化数字人
await this.avatar.init();
// 绑定事件
this.bindEvents();
console.log('数字人系统初始化完成');
}
/**
* 绑定事件
*/
bindEvents() {
// 监听数字人状态变化
this.avatar.on('voiceStateChange', (state) => {
if (state === 'end' && this.isProcessing) {
// 说话完成，恢复待机
this.avatar.idle();
this.isProcessing = false;
}
});
// 监听错误
this.avatar.on('error', (error) => {
console.error('数字人错误:', error);
});
}
/**
* 处理用户输入
*/
async handleUserInput(text, options = {}) {
if (this.isProcessing) {
console.warn('系统正在处理中，请稍候');
return;
}
this.isProcessing = true;
try {
// 1. 切换到倾听状态
this.avatar.listen();
// 2. 切换到思考状态
this.avatar.think();
// 3. 调用AI获取回复
let fullResponse = '';
await this.ai.chat(
text,
(chunk) => {
// 流式处理每个文本块
// 可以在这里实时显示字幕
console.log('AI回复片段:', chunk);
fullResponse += chunk;
},
(complete) => {
// 完整回复接收完成
console.log('完整回复:', complete);
},
(error) => {
console.error('AI错误:', error);
}
);
// 4. 数字人说话
if (options.useVoice !== false) {
await this.avatar.speak(fullResponse);
} else {
this.avatar.idle();
this.isProcessing = false;
}
return fullResponse;
} catch (error) {
console.error('处理失败:', error);
this.avatar.idle();
this.isProcessing = false;
throw error;
}
}
/**
* 开始语音输入
*/
startVoiceInput(onResult) {
this.voice.startListening(
(result) => {
onResult(result);
if (result.final) {
// 语音识别完成，自动提交
this.handleUserInput(result.final);
}
},
(error) => {
console.error('语音识别错误:', error);
}
);
}
/**
* 停止语音输入
*/
stopVoiceInput() {
this.voice.stopListening();
}
/**
* 切换场景
*/
switchScene(sceneConfig) {
// 更新AI的系统提示词
this.ai.updateSystemPrompt(sceneConfig.systemPrompt);
console.log('场景已切换:', sceneConfig.name);
}
/**
* 销毁系统
*/
destroy() {
this.avatar.destroy();
this.voice.stopSpeaking();
this.voice.stopListening();
}
}完整代码示例
HTML结构
html
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>数字人交互系统</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<div class="app-container">
<!-- 顶部导航 -->
<header class="header">
<h1>智能政务服务大厅</h1>
<div class="controls">
<button id="btn-connect" class="btn primary">连接数字人</button>
<button id="btn-disconnect" class="btn danger" disabled>断开连接</button>
<button id="btn-settings" class="btn secondary">设置</button>
</div>
</header>
<!-- 主内容区 -->
<main class="main-content">
<!-- 数字人渲染区 -->
<div class="avatar-section">
<div id="avatar-container" class="avatar-container">
<div class="avatar-placeholder">
<p>点击"连接数字人"开始体验</p>
</div>
</div>
<div class="avatar-status">
<span class="status-indicator" id="status-indicator"></span>
<span id="status-text">未连接</span>
</div>
</div>
<!-- 对话交互区 -->
<div class="chat-section">
<div class="chat-messages" id="chat-messages">
<div class="message system">
<div class="message-content">
你好！我是智能政务助手小政，有什么可以帮您的吗？
</div>
</div>
</div>
<div class="quick-questions">
<button class="quick-btn">如何办理身份证？</button>
<button class="quick-btn">社保查询流程</button>
<button class="quick-btn">不动产登记需要什么材料？</button>
</div>
<div class="input-area">
<textarea id="user-input" placeholder="输入您的问题..."></textarea>
<div class="input-actions">
<button id="btn-voice" class="btn-icon" title="语音输入">
<svg>...</svg>
</button>
<button id="btn-send" class="btn primary">发送</button>
</div>
</div>
</div>
</main>
</div>
<!-- 设置弹窗 -->
<div id="settings-modal" class="modal">
<div class="modal-content">
<h2>系统设置</h2>
<div class="form-group">
<label>数字人AppId</label>
<input type="text" id="input-appid" placeholder="请输入AppId">
</div>
<div class="form-group">
<label>数字人AppSecret</label>
<input type="password" id="input-secret" placeholder="请输入AppSecret">
</div>
<div class="form-group">
<label>AI API Key</label>
<input type="password" id="input-apikey" placeholder="请输入API Key">
</div>
<div class="modal-actions">
<button id="btn-save-settings" class="btn primary">保存</button>
<button id="btn-close-settings" class="btn secondary">关闭</button>
</div>
</div>
</div>
<script src="https://your-cdn.com/avatar-sdk.js"></script>
<script src="js/app.js"></script>
</body>
</html>CSS样式
css
/* 全局样式 */
:root {
--primary-color: #1890ff;
--success-color: #52c41a;
--danger-color: #ff4d4f;
--text-color: #333;
--bg-color: #f0f2f5;
--card-bg: #ffffff;
--border-radius: 8px;
--shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: var(--bg-color);
color: var(--text-color);
height: 100vh;
overflow: hidden;
}
.app-container {
display: flex;
flex-direction: column;
height: 100%;
}
/* 顶部导航 */
.header {
background: var(--card-bg);
padding: 16px 24px;
display: flex;
justify-content: space-between;
align-items: center;
box-shadow: var(--shadow);
z-index: 10;
}
.header h1 {
font-size: 20px;
font-weight: 600;
}
.controls {
display: flex;
gap: 12px;
}
/* 按钮样式 */
.btn {
padding: 8px 16px;
border: none;
border-radius: var(--border-radius);
cursor: pointer;
font-size: 14px;
transition: all 0.3s;
}
.btn.primary {
background: var(--primary-color);
color: white;
}
.btn.primary:hover {
background: #40a9ff;
}
.btn.danger {
background: var(--danger-color);
color: white;
}
.btn.secondary {
background: #f5f5f5;
color: var(--text-color);
}
.btn:disabled {
opacity: 0.5;
cursor: not-allowed;
}
/* 主内容区 */
.main-content {
flex: 1;
display: flex;
gap: 24px;
padding: 24px;
overflow: hidden;
}
/* 数字人区域 */
.avatar-section {
flex: 1;
background: var(--card-bg);
border-radius: var(--border-radius);
box-shadow: var(--shadow);
display: flex;
flex-direction: column;
}
.avatar-container {
flex: 1;
position: relative;
background: #000;
border-radius: var(--border-radius) var(--border-radius) 0 0;
overflow: hidden;
}
.avatar-placeholder {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
display: flex;
align-items: center;
justify-content: center;
color: #666;
}
.avatar-status {
padding: 12px 16px;
display: flex;
align-items: center;
gap: 8px;
border-top: 1px solid #eee;
}
.status-indicator {
width: 8px;
height: 8px;
border-radius: 50%;
background: #ccc;
}
.status-indicator.connected {
background: var(--success-color);
}
.status-indicator.connecting {
background: var(--primary-color);
animation: pulse 1s infinite;
}
@keyframes pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.5; }
}
/* 对话区域 */
.chat-section {
flex: 1;
max-width: 500px;
background: var(--card-bg);
border-radius: var(--border-radius);
box-shadow: var(--shadow);
display: flex;
flex-direction: column;
}
.chat-messages {
flex: 1;
overflow-y: auto;
padding: 16px;
display: flex;
flex-direction: column;
gap: 12px;
}
.message {
max-width: 80%;
}
.message.user {
align-self: flex-end;
}
.message.assistant {
align-self: flex-start;
}
.message.system {
align-self: center;
}
.message-content {
padding: 12px 16px;
border-radius: 12px;
line-height: 1.5;
}
.message.user .message-content {
background: var(--primary-color);
color: white;
}
.message.assistant .message-content {
background: #f5f5f5;
color: var(--text-color);
}
.message.system .message-content {
background: #e6f7ff;
color: #1890ff;
font-size: 12px;
}
/* 快捷问题 */
.quick-questions {
padding: 12px 16px;
display: flex;
flex-wrap: wrap;
gap: 8px;
border-top: 1px solid #eee;
}
.quick-btn {
padding: 6px 12px;
background: #f0f2f5;
border: none;
border-radius: 16px;
font-size: 12px;
cursor: pointer;
transition: background 0.3s;
}
.quick-btn:hover {
background: #d9d9d9;
}
/* 输入区域 */
.input-area {
padding: 16px;
border-top: 1px solid #eee;
display: flex;
flex-direction: column;
gap: 12px;
}
#user-input {
width: 100%;
min-height: 60px;
padding: 12px;
border: 1px solid #ddd;
border-radius: var(--border-radius);
resize: none;
font-family: inherit;
}
#user-input:focus {
outline: none;
border-color: var(--primary-color);
}
.input-actions {
display: flex;
justify-content: space-between;
align-items: center;
}
.btn-icon {
width: 36px;
height: 36px;
border: none;
background: #f5f5f5;
border-radius: 50%;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
}
.btn-icon:hover {
background: #e6e6e6;
}
.btn-icon.recording {
background: var(--danger-color);
color: white;
animation: pulse 1s infinite;
}
/* 设置弹窗 */
.modal {
display: none;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
align-items: center;
justify-content: center;
z-index: 1000;
}
.modal.active {
display: flex;
}
.modal-content {
background: white;
padding: 24px;
border-radius: var(--border-radius);
width: 400px;
max-width: 90%;
}
.modal-content h2 {
margin-bottom: 20px;
font-size: 18px;
}
.form-group {
margin-bottom: 16px;
}
.form-group label {
display: block;
margin-bottom: 8px;
font-size: 14px;
font-weight: 500;
}
.form-group input {
width: 100%;
padding: 8px 12px;
border: 1px solid #ddd;
border-radius: 4px;
}
.modal-actions {
display: flex;
justify-content: flex-end;
gap: 12px;
margin-top: 20px;
}
/* 响应式设计 */
@media (max-width: 768px) {
.main-content {
flex-direction: column;
overflow-y: auto;
}
.avatar-section {
min-height: 300px;
}
.chat-section {
max-width: 100%;
min-height: 400px;
}
}应用入口
javascript
// 配置管理
const ConfigManager = {
STORAGE_KEY: 'digital_human_config',
save(config) {
localStorage.setItem(this.STORAGE_KEY, JSON.stringify(config));
},
load() {
const data = localStorage.getItem(this.STORAGE_KEY);
return data ? JSON.parse(data) : null;
},
clear() {
localStorage.removeItem(this.STORAGE_KEY);
}
};
// UI管理器
class UIManager {
constructor() {
this.elements = {
messagesContainer: document.getElementById('chat-messages'),
userInput: document.getElementById('user-input'),
statusIndicator: document.getElementById('status-indicator'),
statusText: document.getElementById('status-text'),
btnConnect: document.getElementById('btn-connect'),
btnDisconnect: document.getElementById('btn-disconnect'),
btnSend: document.getElementById('btn-send'),
btnVoice: document.getElementById('btn-voice')
};
}
addMessage(content, type = 'assistant') {
const messageDiv = document.createElement('div');
messageDiv.className = `message ${type}`;
const contentDiv = document.createElement('div');
contentDiv.className = 'message-content';
contentDiv.textContent = content;
messageDiv.appendChild(contentDiv);
this.elements.messagesContainer.appendChild(messageDiv);
// 滚动到底部
this.elements.messagesContainer.scrollTop =
this.elements.messagesContainer.scrollHeight;
}
clearMessages() {
this.elements.messagesContainer.innerHTML = '';
}
updateStatus(status, text) {
this.elements.statusIndicator.className = `status-indicator ${status}`;
this.elements.statusText.textContent = text;
}
setConnecting(isConnecting) {
this.elements.btnConnect.disabled = isConnecting;
this.elements.btnDisconnect.disabled = !isConnecting;
}
}
// 主应用
class App {
constructor() {
this.system = null;
this.ui = new UIManager();
this.config = ConfigManager.load() || this.getDefaultConfig();
this.initUI();
}
getDefaultConfig() {
return {
avatarAppId: '',
avatarAppSecret: '',
aiApiKey: '',
aiModel: 'qwen-plus'
};
}
initUI() {
// 绑定按钮事件
this.ui.elements.btnConnect.addEventListener('click', () => this.connect());
this.ui.elements.btnDisconnect.addEventListener('click', () => this.disconnect());
this.ui.elements.btnSend.addEventListener('click', () => this.sendMessage());
this.ui.elements.btnVoice.addEventListener('click', () => this.toggleVoice());
// 输入框回车发送
this.ui.elements.userInput.addEventListener('keydown', (e) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
this.sendMessage();
}
});
// 快捷问题
document.querySelectorAll('.quick-btn').forEach(btn => {
btn.addEventListener('click', () => {
this.ui.elements.userInput.value = btn.textContent;
this.sendMessage();
});
});
}
async connect() {
if (!this.validateConfig()) {
alert('请先配置API密钥');
return;
}
this.ui.updateStatus('connecting', '连接中...');
this.ui.setConnecting(true);
try {
this.system = new DigitalHumanSystem(this.config);
await this.system.init();
this.ui.updateStatus('connected', '已连接');
this.ui.addMessage('数字人已连接，可以开始对话了！', 'system');
} catch (error) {
console.error('连接失败:', error);
this.ui.updateStatus('', '连接失败');
alert('连接失败: ' + error.message);
this.ui.setConnecting(false);
}
}
disconnect() {
if (this.system) {
this.system.destroy();
this.system = null;
}
this.ui.updateStatus('', '未连接');
this.ui.setConnecting(false);
}
async sendMessage() {
const input = this.ui.elements.userInput.value.trim();
if (!input || !this.system) return;
// 显示用户消息
this.ui.addMessage(input, 'user');
this.ui.elements.userInput.value = '';
try {
// 处理并获取AI回复
const response = await this.system.handleUserInput(input);
this.ui.addMessage(response, 'assistant');
} catch (error) {
console.error('发送消息失败:', error);
this.ui.addMessage('抱歉，处理您的请求时出现错误。', 'system');
}
}
toggleVoice() {
const btn = this.ui.elements.btnVoice;
if (btn.classList.contains('recording')) {
// 停止录音
this.system?.stopVoiceInput();
btn.classList.remove('recording');
} else {
// 开始录音
this.system?.startVoiceInput((result) => {
this.ui.elements.userInput.value = result.final || result.interim;
});
btn.classList.add('recording');
}
}
validateConfig() {
return this.config.avatarAppId &&
this.config.avatarAppSecret &&
this.config.aiApiKey;
}
}
// 启动应用
document.addEventListener('DOMContentLoaded', () => {
window.app = new App();
});最佳实践与优化
1. 性能优化
javascript
// 对话历史管理
class ConversationManager {
constructor(maxHistory = 20) {
this.maxHistory = maxHistory;
this.history = [];
}
addMessage(role, content) {
this.history.push({ role, content });
// 超过限制时删除旧消息
if (this.history.length > this.maxHistory) {
this.history = this.history.slice(-this.maxHistory);
}
}
// 计算token数量，避免超出模型限制
estimateTokens(text) {
// 粗略估计：中文约1.5字符/token，英文约4字符/token
const chineseChars = (text.match(/[\u4e00-\u9fa5]/g) || []).length;
const otherChars = text.length - chineseChars;
return Math.ceil(chineseChars / 1.5 + otherChars / 4);
}
trimToTokenLimit(maxTokens) {
let totalTokens = 0;
const trimmedHistory = [];
for (let i = this.history.length - 1; i >= 0; i--) {
const tokens = this.estimateTokens(this.history[i].content);
if (totalTokens + tokens > maxTokens) break;
trimmedHistory.unshift(this.history[i]);
totalTokens += tokens;
}
this.history = trimmedHistory;
}
}2. 错误处理与重试
javascript
class RetryableRequest {
constructor(maxRetries = 3, baseDelay = 1000) {
this.maxRetries = maxRetries;
this.baseDelay = baseDelay;
}
async execute(requestFn) {
let lastError;
for (let attempt = 0; attempt < this.maxRetries; attempt++) {
try {
return await requestFn();
} catch (error) {
lastError = error;
// 判断是否可重试
if (!this.isRetryable(error)) {
throw error;
}
// 指数退避
const delay = this.baseDelay * Math.pow(2, attempt);
await this.sleep(delay);
}
}
throw lastError;
}
isRetryable(error) {
// 429 Too Many Requests
// 500 Internal Server Error
// 502 Bad Gateway
// 503 Service Unavailable
const retryableStatuses = [429, 500, 502, 503];
return retryableStatuses.includes(error.status);
}
sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
}
// 使用示例
const requester = new RetryableRequest();
const response = await requester.execute(() =>
fetch('https://api.example.com/data')
);3. 状态同步
javascript
// 使用状态机管理复杂交互
class AvatarStateMachine {
constructor() {
this.states = {
IDLE: 'idle',
LISTEN: 'listen',
THINK: 'think',
SPEAK: 'speak'
};
this.transitions = {
[this.states.IDLE]: [this.states.LISTEN],
[this.states.LISTEN]: [this.states.THINK, this.states.IDLE],
[this.states.THINK]: [this.states.SPEAK, this.states.IDLE],
[this.states.SPEAK]: [this.states.IDLE, this.states.LISTEN]
};
this.currentState = this.states.IDLE;
this.observers = [];
}
canTransition(newState) {
return this.transitions[this.currentState]?.includes(newState);
}
transition(newState) {
if (!this.canTransition(newState)) {
throw new Error(`不能从 ${this.currentState} 转换到 ${newState}`);
}
const oldState = this.currentState;
this.currentState = newState;
this.notifyObservers({ oldState, newState });
}
subscribe(observer) {
this.observers.push(observer);
}
notifyObservers(event) {
this.observers.forEach(observer => observer(event));
}
}4. 资源预加载
javascript
class ResourcePreloader {
constructor() {
this.loadedResources = new Set();
}
async preloadImages(urls) {
const promises = urls.map(url => {
return new Promise((resolve, reject) => {
if (this.loadedResources.has(url)) {
resolve();
return;
}
const img = new Image();
img.onload = () => {
this.loadedResources.add(url);
resolve();
};
img.onerror = reject;
img.src = url;
});
});
return Promise.all(promises);
}
async preloadAudio(urls) {
const promises = urls.map(url => {
return new Promise((resolve, reject) => {
if (this.loadedResources.has(url)) {
resolve();
return;
}
const audio = new Audio();
audio.oncanplaythrough = () => {
this.loadedResources.add(url);
resolve();
};
audio.onerror = reject;
audio.src = url;
});
});
return Promise.all(promises);
}
}总结与展望
技术要点总结
本文介绍了从零构建数字人系统的完整流程，核心要点包括：
- 架构设计：模块化设计，分离数字人控制、AI对话、语音处理等职责
- 状态管理：使用状态机管理数字人的各种状态和转换
- 流式交互：实现AI回复的流式输出，提升用户体验
- 错误处理：完善的重试机制和错误边界处理
- 性能优化：资源预加载、对话历史管理等优化手段
技术演进方向
当前阶段
增强交互
多模态融合
边缘部署
情感识别
手势交互
眼神追踪
视觉理解
声音克隆
场景感知
WebGL加速
本地模型
离线运行
未来展望
数字人技术正朝着以下方向发展：
- 更自然的交互：结合情感计算、手势识别，实现更接近真人的交互体验
- 多模态融合：视觉、听觉、触觉的多模态感知和表达
- 个性化定制：根据用户喜好调整数字人的外观、声音、性格
- 边缘计算：通过WebGL、WebGPU等技术实现浏览器端的高性能渲染
- 行业深化：在政务、金融、教育、医疗等领域的深度应用
参考资源
如果这篇文章对你有帮助，欢迎点赞、收藏、分享！有问题或建议，欢迎在评论区交流讨论。