HarmonyOS 6（API 23）实战：基于 Face AR 疼痛评估与 Body AR 姿态追踪的“智能康复训练助手“

文章目录

- 每日一句正能量
- 前言
- 一、康复医疗的数字化困境与AR破局
- - [1.1 传统康复训练的核心痛点](#1.1 传统康复训练的核心痛点)
  - [1.2 "智能康复训练助手"系统架构](#1.2 "智能康复训练助手"系统架构)
- 二、环境配置与系统初始化
- - [2.1 模块依赖配置](#2.1 模块依赖配置)
  - [2.2 康复训练窗口配置（RehabAbility.ets）](#2.2 康复训练窗口配置（RehabAbility.ets）)
- 三、核心组件实战
- - [3.1 Face AR 疼痛评估引擎（PainAssessmentEngine.ets）](#3.1 Face AR 疼痛评估引擎（PainAssessmentEngine.ets）)
  - [3.2 Body AR 康复动作追踪器（RehabMotionTracker.ets）](#3.2 Body AR 康复动作追踪器（RehabMotionTracker.ets）)
  - [3.3 康复训练主界面（RehabTrainingPage.ets）](#3.3 康复训练主界面（RehabTrainingPage.ets）)
- 四、关键技术总结
- - [4.1 Face AR 疼痛评估体系](#4.1 Face AR 疼痛评估体系)
  - [4.2 Body AR 康复动作评估维度](#4.2 Body AR 康复动作评估维度)
  - [4.3 沉浸光效与疼痛状态联动](#4.3 沉浸光效与疼痛状态联动)
- 五、调试与部署建议
- - [5.1 调试要点](#5.1 调试要点)
  - [5.2 部署场景](#5.2 部署场景)
- 六、总结与展望

每日一句正能量

心可以碎，手不能停，该干什么干什么，在崩溃中继续前行，这才是一个成年人的素养。

允许情绪宣泄，但绝不放弃行动。这正是"成年人的素养"：在崩溃中维持秩序，在伤痛中坚守责任。

前言

摘要：康复训练是术后恢复和运动损伤治疗的核心环节，但传统康复训练长期面临"动作不标准无人纠正、疼痛程度难以量化、训练数据无法追溯"三大痛点。HarmonyOS 6（API 23）的 Face AR 与 Body AR 能力为康复医疗带来了数字化革新------通过 Face AR 实时识别患者面部微表情评估疼痛等级，通过 Body AR 精确追踪康复动作的角度和幅度，结合悬浮导航与沉浸光感，打造"看得懂疼痛、纠得正动作"的智能康复训练系统。

一、康复医疗的数字化困境与AR破局

1.1 传统康复训练的核心痛点

痛点场景	传统解决方案	问题
动作不标准	康复师每周1-2次现场指导	间隔长、记忆衰减、错误动作固化
疼痛难量化	患者主观描述（1-10分）	个体差异大、记忆偏差、缺乏客观依据
进度难追踪	纸质记录或简单表格	数据离散、趋势难分析、无法远程共享
依从性低	口头叮嘱	患者孤独感强、缺乏即时反馈、容易放弃

HarmonyOS 6（API 23）的 AR Engine 6.1.0 提供了 Face AR （64种BlendShape表情参数、疼痛相关微表情识别）和 Body AR（20+骨骼关键点、关节角度精确计算）两大核心能力，为康复训练提供了"数字化康复师"------24小时在线、精准量化、即时反馈。

1.2 "智能康复训练助手"系统架构

复制代码

┌─────────────────────────────────────────────────────────────┐
│                    患者端（手机/平板/PC）                      │
│  ┌──────────────────────────────────────────────────────┐   │
│  │              康复动作演示区（3D骨骼模型）               │   │
│  │  · 标准动作轨迹叠加显示                                │   │
│  │  · 实时偏差高亮提示（红色=超标、绿色=达标、黄色=接近）   │   │
│  │  · 关节角度数值实时显示                                │   │
│  └──────────────────────────────────────────────────────┘   │
│                          ↑                                  │
│              AR实时追踪层（Face + Body双模态）                 │
│  ┌─────────────────────┐    ┌─────────────────────────────┐ │
│  │   Face AR 疼痛评估   │    │     Body AR 姿态追踪        │ │
│  │  · 眉头紧锁检测      │    │  · 关节角度计算             │ │
│  │  · 嘴角下拉识别      │    │  · 动作轨迹记录             │ │
│  │  · 眼睛眯起分析      │    │  · 对称性评估               │ │
│  │  · 疼痛等级0-10输出  │    │  · 活动范围ROM测量          │ │
│  └─────────────────────┘    └─────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                              │
        ┌─────────────────────┼─────────────────────┐
        ↓                     ↓                     ↓
   ┌─────────┐          ┌─────────┐          ┌─────────┐
   │ 康复师端 │          │ 家属端   │          │ 云端档案 │
   │ PC大屏   │          │ 手机    │          │ 区块链   │
   │ 查看报告 │          │ 接收提醒│          │ 存证   │
   └─────────┘          └─────────┘          └─────────┘

二、环境配置与系统初始化

2.1 模块依赖配置

json 复制代码

// oh-package.json5
{
  "dependencies": {
    "@hms.core.ar.arengine": "^6.1.0",
    "@kit.ArkUI": "^6.1.0",
    "@kit.AbilityKit": "^6.1.0",
    "@kit.SensorServiceKit": "^6.1.0",
    "@kit.HealthServiceKit": "^6.1.0"
  }
}

2.2 康复训练窗口配置（RehabAbility.ets）

代码亮点：康复训练需要大屏幕展示3D骨骼模型和动作轨迹，配置为PC端自由窗口模式，支持患者家属陪同观看。窗口背景采用低刺激色调，避免干扰患者注意力。

typescript 复制代码

// entry/src/main/ets/ability/RehabAbility.ets
import { AbilityConstant, UIAbility, Want } from '@kit.AbilityKit';
import { window } from '@kit.ArkUI';

/**
 * 康复训练配置
 */
interface RehabConfig {
  patientId: string;
  rehabPlanId: string;
  targetJoint: string;      // 目标关节（如左肩、右膝）
  targetROM: { min: number; max: number };  // 目标活动范围
  painThreshold: number;    // 疼痛阈值（超过则暂停）
  sessionDuration: number;  // 单次训练时长（秒）
  restInterval: number;     // 休息间隔（秒）
}

export default class RehabAbility extends UIAbility {
  private mainWindow: window.Window | null = null;
  private rehabConfig: RehabConfig = {
    patientId: 'P202604001',
    rehabPlanId: 'RP_SHOULDER_001',
    targetJoint: 'left_shoulder',
    targetROM: { min: 0, max: 180 },
    painThreshold: 7,
    sessionDuration: 300,
    restInterval: 60
  };

  onWindowStageCreate(windowStage: window.WindowStage): void {
    this.setupRehabWindow(windowStage);
  }

  /**
   * 配置康复训练窗口：大屏、低刺激、高对比度
   */
  private async setupRehabWindow(windowStage: window.WindowStage): Promise<void> {
    try {
      this.mainWindow = windowStage.getMainWindowSync();
      
      // PC端自由窗口：适合大屏展示
      await this.mainWindow.setWindowSizeType(window.WindowSizeType.FREE);
      await this.mainWindow.setWindowMode(window.WindowMode.FULLSCREEN);
      await this.mainWindow.setWindowTitleBarEnable(false);
      await this.mainWindow.setWindowBackgroundColor('#0F1419');  // 深蓝灰，低刺激
      await this.mainWindow.setWindowLayoutFullScreen(true);

      // 禁用系统手势，防止训练过程中误触
      await this.mainWindow.setWindowGestureDisabled(true);

      // 高刷新率：确保动作追踪画面流畅
      try {
        await this.mainWindow.setPreferredFrameRate(120);
      } catch (e) {
        await this.mainWindow.setPreferredFrameRate(60);
      }

      AppStorage.setOrCreate('rehab_config', this.rehabConfig);
      AppStorage.setOrCreate('main_window', this.mainWindow);

      windowStage.loadContent('pages/RehabTrainingPage', (err) => {
        if (err.code) {
          console.error('Failed to load rehab page:', JSON.stringify(err));
        }
      });

    } catch (error) {
      console.error('Rehab window setup failed:', error);
    }
  }

  onWindowStageDestroy(): void {
    // 保存训练数据到健康服务
  }
}

三、核心组件实战

3.1 Face AR 疼痛评估引擎（PainAssessmentEngine.ets）

代码亮点：基于 Face AR 的 BlendShape 参数，构建疼痛表情识别模型。疼痛表情与日常表情有显著差异：眉头紧锁（BROW_DOWN）、嘴角下拉（MOUTH_FROWN）、眼睛眯起（EYE_SQUINT）、面部肌肉紧张（CHEEK_PUFF）等。通过多参数加权融合，输出0-10的疼痛等级。

typescript 复制代码

// utils/PainAssessmentEngine.ets
import { arEngine } from '@hms.core.ar.arengine';

/**
 * 疼痛评估结果
 */
export interface PainAssessment {
  level: number;           // 0-10 疼痛等级
  confidence: number;      // 置信度 0-1
  indicators: {            // 分项指标
    browTension: number;    // 眉头紧张度
    mouthTension: number;   // 嘴部紧张度
    eyeTension: number;     // 眼部紧张度
    jawTension: number;     // 下颌紧张度
  };
  trend: 'rising' | 'stable' | 'falling';  // 趋势
  timestamp: number;
}

/**
 * 疼痛评估引擎
 */
export class PainAssessmentEngine {
  private static instance: PainAssessmentEngine;
  private history: PainAssessment[] = [];
  private readonly HISTORY_SIZE = 30;  // 保留最近30帧

  static getInstance(): PainAssessmentEngine {
    if (!PainAssessmentEngine.instance) {
      PainAssessmentEngine.instance = new PainAssessmentEngine();
    }
    return PainAssessmentEngine.instance;
  }

  /**
   * 评估疼痛等级
   */
  assessPain(faceAnchor: arEngine.ARFaceAnchor): PainAssessment {
    const face = faceAnchor.getFace();
    if (!face) return this.getDefaultAssessment();

    const blendShapes = face.getBlendShapes();
    const data = new Float32Array(blendShapes.getData());
    const types = blendShapes.getTypes();
    
    const expressions = new Map<string, number>();
    types.forEach((type, i) => expressions.set(type.toString(), data[i]));

    // 疼痛相关表情参数提取
    const indicators = {
      browTension: this.calculateBrowTension(expressions),
      mouthTension: this.calculateMouthTension(expressions),
      eyeTension: this.calculateEyeTension(expressions),
      jawTension: this.calculateJawTension(expressions)
    };

    // 加权融合计算疼痛等级
    const weights = { brow: 0.35, mouth: 0.25, eye: 0.25, jaw: 0.15 };
    const level = Math.min(10, Math.round(
      indicators.browTension * weights.brow +
      indicators.mouthTension * weights.mouth +
      indicators.eyeTension * weights.eye +
      indicators.jawTension * weights.jaw
    ));

    // 计算置信度（基于参数强度）
    const confidence = Math.min(1.0, 
      (indicators.browTension + indicators.mouthTension + 
       indicators.eyeTension + indicators.jawTension) / 40
    );

    const assessment: PainAssessment = {
      level,
      confidence,
      indicators,
      trend: this.calculateTrend(level),
      timestamp: Date.now()
    };

    this.history.push(assessment);
    if (this.history.length > this.HISTORY_SIZE) {
      this.history.shift();
    }

    return assessment;
  }

  /**
   * 计算眉头紧张度（疼痛核心指标）
   * 疼痛时眉头紧锁、眉间出现皱纹
   */
  private calculateBrowTension(expressions: Map<string, number>): number {
    const browDown = expressions.get('BROW_DOWN_LEFT') || 0;
    const browDownRight = expressions.get('BROW_DOWN_RIGHT') || 0;
    const browInnerUp = expressions.get('BROW_INNER_UP') || 0;  // 痛苦时内眉上扬
    
    // 眉头下压 + 内眉上扬 = 典型疼痛表情
    const tension = (browDown + browDownRight) * 0.6 + browInnerUp * 0.4;
    return Math.min(10, tension * 12);  // 放大到0-10范围
  }

  /**
   * 计算嘴部紧张度
   * 疼痛时嘴角下拉、嘴唇抿紧
   */
  private calculateMouthTension(expressions: Map<string, number>): number {
    const mouthFrown = expressions.get('MOUTH_FROWN_LEFT') || 0;
    const mouthFrownRight = expressions.get('MOUTH_FROWN_RIGHT') || 0;
    const mouthPucker = expressions.get('MOUTH_PUCKER') || 0;  // 抿嘴
    
    const tension = (mouthFrown + mouthFrownRight) * 0.5 + mouthPucker * 0.5;
    return Math.min(10, tension * 14);
  }

  /**
   * 计算眼部紧张度
   * 疼痛时眼睛眯起、眨眼减少（凝视）
   */
  private calculateEyeTension(expressions: Map<string, number>): number {
    const eyeSquint = expressions.get('EYE_SQUINT_LEFT') || 0;
    const eyeSquintRight = expressions.get('EYE_SQUINT_RIGHT') || 0;
    const eyeWide = expressions.get('EYE_WIDE_LEFT') || 0;  // 惊讶/疼痛时睁大
    
    // 眯眼或异常睁大都可能表示疼痛
    const tension = Math.max(eyeSquint, eyeSquintRight) * 0.7 + eyeWide * 0.3;
    return Math.min(10, tension * 12);
  }

  /**
   * 计算下颌紧张度
   * 疼痛时咬牙、下颌僵硬
   */
  private calculateJawTension(expressions: Map<string, number>): number {
    const jawForward = expressions.get('JAW_FORWARD') || 0;
    const jawClench = expressions.get('JAW_CLENCH') || 0;  // 咬牙
    
    const tension = jawForward * 0.3 + jawClench * 0.7;
    return Math.min(10, tension * 15);
  }

  /**
   * 计算疼痛趋势
   */
  private calculateTrend(currentLevel: number): 'rising' | 'stable' | 'falling' {
    if (this.history.length < 5) return 'stable';
    
    const recent = this.history.slice(-5);
    const avg = recent.reduce((sum, h) => sum + h.level, 0) / recent.length;
    
    if (currentLevel > avg + 1.5) return 'rising';
    if (currentLevel < avg - 1.5) return 'falling';
    return 'stable';
  }

  /**
   * 获取疼痛历史统计
   */
  getPainStatistics(): {
    average: number;
    peak: number;
    duration: number;  // 疼痛持续时间（秒）
    trend: string;
  } {
    if (this.history.length === 0) {
      return { average: 0, peak: 0, duration: 0, trend: 'stable' };
    }

    const levels = this.history.map(h => h.level);
    const significantPain = levels.filter(l => l > 3);
    
    return {
      average: Math.round(levels.reduce((a, b) => a + b, 0) / levels.length),
      peak: Math.max(...levels),
      duration: significantPain.length * 2,  // 假设每帧2秒
      trend: this.history[this.history.length - 1]?.trend || 'stable'
    };
  }

  private getDefaultAssessment(): PainAssessment {
    return {
      level: 0,
      confidence: 0,
      indicators: { browTension: 0, mouthTension: 0, eyeTension: 0, jawTension: 0 },
      trend: 'stable',
      timestamp: Date.now()
    };
  }

  reset(): void {
    this.history = [];
  }
}

3.2 Body AR 康复动作追踪器（RehabMotionTracker.ets）

代码亮点：基于 Body AR 的20+骨骼关键点，精确计算康复动作中的关节角度、活动范围（ROM）、对称性和动作速度。与标准康复动作轨迹对比，实时给出偏差提示。

typescript 复制代码

// utils/RehabMotionTracker.ets
import { arEngine } from '@hms.core.ar.arengine';

/**
 * 关节角度数据
 */
export interface JointAngle {
  joint: string;
  angle: number;           // 当前角度
  targetMin: number;       // 目标最小角度
  targetMax: number;       // 目标最大角度
  deviation: number;       // 偏差百分比
  status: 'perfect' | 'good' | 'warning' | 'danger';  // 状态
}

/**
 * 动作轨迹点
 */
export interface MotionPoint {
  timestamp: number;
  jointAngles: JointAngle[];
  bodyPosition: { x: number; y: number; z: number };
  stability: number;       // 身体稳定性 0-1
}

/**
 * 康复动作评估
 */
export interface RehabAssessment {
  romScore: number;        // 活动范围评分 0-100
  symmetryScore: number;   // 对称性评分 0-100
  stabilityScore: number;    // 稳定性评分 0-100
  speedScore: number;      // 速度评分 0-100
  overallScore: number;      // 综合评分 0-100
  recommendations: string[]; // 改进建议
}

export class RehabMotionTracker {
  private static instance: RehabMotionTracker;
  private motionHistory: MotionPoint[] = [];
  private standardTrajectory: MotionPoint[] = [];  // 标准动作轨迹
  private readonly HISTORY_SIZE = 60;  // 保留60帧（约2秒@30fps）

  static getInstance(): RehabMotionTracker {
    if (!RehabMotionTracker.instance) {
      RehabMotionTracker.instance = new RehabMotionTracker();
    }
    return RehabMotionTracker.instance;
  }

  /**
   * 加载标准康复动作轨迹（由康复师录制）
   */
  loadStandardTrajectory(trajectory: MotionPoint[]): void {
    this.standardTrajectory = trajectory;
  }

  /**
   * 追踪康复动作
   */
  trackMotion(body: arEngine.ARBody): MotionPoint {
    const landmarks = body.getLandmarks3D();  // 使用3D关键点更精确
    
    // 计算关键关节角度
    const jointAngles = this.calculateJointAngles(landmarks);
    
    // 计算身体稳定性
    const stability = this.calculateStability(landmarks);
    
    const point: MotionPoint = {
      timestamp: Date.now(),
      jointAngles,
      bodyPosition: this.getBodyCenter(landmarks),
      stability
    };

    this.motionHistory.push(point);
    if (this.motionHistory.length > this.HISTORY_SIZE) {
      this.motionHistory.shift();
    }

    return point;
  }

  /**
   * 计算关节角度（核心算法）
   */
  private calculateJointAngles(landmarks: ArrayBuffer): JointAngle[] {
    const floatView = new Float32Array(landmarks);
    const angles: JointAngle[] = [];

    // 肩关节角度（左肩）
    const leftShoulder = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_SHOULDER);
    const leftElbow = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_ELBOW);
    const leftWrist = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_WRIST);
    
    if (leftShoulder && leftElbow && leftWrist) {
      const shoulderAngle = this.calculate3DAngle(leftShoulder, leftElbow, leftWrist);
      angles.push({
        joint: 'left_shoulder',
        angle: shoulderAngle,
        targetMin: 0,
        targetMax: 180,
        deviation: this.calculateDeviation(shoulderAngle, 0, 180),
        status: this.getStatus(shoulderAngle, 0, 180)
      });
    }

    // 肩关节角度（右肩）
    const rightShoulder = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_SHOULDER);
    const rightElbow = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_ELBOW);
    const rightWrist = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_WRIST);
    
    if (rightShoulder && rightElbow && rightWrist) {
      const shoulderAngle = this.calculate3DAngle(rightShoulder, rightElbow, rightWrist);
      angles.push({
        joint: 'right_shoulder',
        angle: shoulderAngle,
        targetMin: 0,
        targetMax: 180,
        deviation: this.calculateDeviation(shoulderAngle, 0, 180),
        status: this.getStatus(shoulderAngle, 0, 180)
      });
    }

    // 髋关节角度（左髋）
    const leftHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_HIP);
    const leftKnee = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_KNEE);
    const leftAnkle = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_ANKLE);
    
    if (leftHip && leftKnee && leftAnkle) {
      const hipAngle = this.calculate3DAngle(leftHip, leftKnee, leftAnkle);
      angles.push({
        joint: 'left_hip',
        angle: hipAngle,
        targetMin: 0,
        targetMax: 120,
        deviation: this.calculateDeviation(hipAngle, 0, 120),
        status: this.getStatus(hipAngle, 0, 120)
      });
    }

    // 膝关节角度（右膝）
    const rightHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_HIP);
    const rightKnee = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_KNEE);
    const rightAnkle = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_ANKLE);
    
    if (rightHip && rightKnee && rightAnkle) {
      const kneeAngle = this.calculate3DAngle(rightHip, rightKnee, rightAnkle);
      angles.push({
        joint: 'right_knee',
        angle: kneeAngle,
        targetMin: 0,
        targetMax: 135,
        deviation: this.calculateDeviation(kneeAngle, 0, 135),
        status: this.getStatus(kneeAngle, 0, 135)
      });
    }

    return angles;
  }

  /**
   * 3D角度计算（余弦定理）
   */
  private calculate3DAngle(
    p1: { x: number; y: number; z: number },
    p2: { x: number; y: number; z: number },
    p3: { x: number; y: number; z: number }
  ): number {
    const v1 = { x: p1.x - p2.x, y: p1.y - p2.y, z: p1.z - p2.z };
    const v2 = { x: p3.x - p2.x, y: p3.y - p2.y, z: p3.z - p2.z };
    
    const dot = v1.x * v2.x + v1.y * v2.y + v1.z * v2.z;
    const mag1 = Math.sqrt(v1.x ** 2 + v1.y ** 2 + v1.z ** 2);
    const mag2 = Math.sqrt(v2.x ** 2 + v2.y ** 2 + v2.z ** 2);
    
    const cos = dot / (mag1 * mag2);
    return Math.round(Math.acos(Math.max(-1, Math.min(1, cos))) * 180 / Math.PI);
  }

  /**
   * 计算偏差百分比
   */
  private calculateDeviation(current: number, min: number, max: number): number {
    const range = max - min;
    if (range === 0) return 0;
    const target = (min + max) / 2;
    return Math.round(Math.abs(current - target) / range * 100);
  }

  /**
   * 获取状态评级
   */
  private getStatus(current: number, min: number, max: number): 'perfect' | 'good' | 'warning' | 'danger' {
    const deviation = this.calculateDeviation(current, min, max);
    if (deviation < 10) return 'perfect';
    if (deviation < 25) return 'good';
    if (deviation < 50) return 'warning';
    return 'danger';
  }

  /**
   * 计算身体稳定性
   */
  private calculateStability(landmarks: ArrayBuffer): number {
    const floatView = new Float32Array(landmarks);
    
    // 计算髋部中心的晃动幅度
    const leftHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_HIP);
    const rightHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_HIP);
    
    if (!leftHip || !rightHip) return 0.5;

    const hipCenter = {
      x: (leftHip.x + rightHip.x) / 2,
      y: (leftHip.y + rightHip.y) / 2,
      z: (leftHip.z + rightHip.z) / 2
    };

    // 与历史位置比较
    if (this.motionHistory.length === 0) return 1.0;

    const lastPoint = this.motionHistory[this.motionHistory.length - 1];
    const displacement = Math.sqrt(
      (hipCenter.x - lastPoint.bodyPosition.x) ** 2 +
      (hipCenter.y - lastPoint.bodyPosition.y) ** 2
    );

    // 位移越小越稳定
    return Math.max(0, 1 - displacement / 50);
  }

  /**
   * 获取身体中心点
   */
  private getBodyCenter(landmarks: ArrayBuffer): { x: number; y: number; z: number } {
    const floatView = new Float32Array(landmarks);
    const leftHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.LEFT_HIP);
    const rightHip = this.getLandmark3D(floatView, arEngine.ARBodyLandmarkType.RIGHT_HIP);
    
    if (!leftHip || !rightHip) return { x: 0, y: 0, z: 0 };
    
    return {
      x: (leftHip.x + rightHip.x) / 2,
      y: (leftHip.y + rightHip.y) / 2,
      z: (leftHip.z + rightHip.z) / 2
    };
  }

  /**
   * 提取3D关键点
   */
  private getLandmark3D(
    floatView: Float32Array,
    type: arEngine.ARBodyLandmarkType
  ): { x: number; y: number; z: number } | null {
    // 简化：假设landmarks数组按type顺序排列，每个点3个float（x,y,z）
    const index = Object.values(arEngine.ARBodyLandmarkType).indexOf(type);
    if (index < 0) return null;
    
    const offset = index * 3;
    if (offset + 2 >= floatView.length) return null;
    
    return {
      x: floatView[offset],
      y: floatView[offset + 1],
      z: floatView[offset + 2]
    };
  }

  /**
   * 综合评估康复动作质量
   */
  assessRehabQuality(): RehabAssessment {
    if (this.motionHistory.length < 10) {
      return {
        romScore: 0, symmetryScore: 0, stabilityScore: 0, speedScore: 0,
        overallScore: 0, recommendations: ['继续训练以获取评估数据']
      };
    }

    // 活动范围评分
    const romScore = this.calculateROMScore();

    // 对称性评分（左右对比）
    const symmetryScore = this.calculateSymmetryScore();

    // 稳定性评分
    const stabilityScore = Math.round(
      this.motionHistory.reduce((sum, p) => sum + p.stability, 0) / 
      this.motionHistory.length * 100
    );

    // 速度评分（与标准轨迹对比）
    const speedScore = this.calculateSpeedScore();

    // 综合评分
    const overallScore = Math.round(
      romScore * 0.35 + symmetryScore * 0.25 + stabilityScore * 0.25 + speedScore * 0.15
    );

    // 生成建议
    const recommendations = this.generateRecommendations(romScore, symmetryScore, stabilityScore);

    return {
      romScore, symmetryScore, stabilityScore, speedScore, overallScore, recommendations
    };
  }

  private calculateROMScore(): number {
    // 计算实际ROM与目标ROM的达成率
    const targetJoint = AppStorage.get<string>('target_joint') || 'left_shoulder';
    const targetROM = AppStorage.get<{ min: number; max: number }>('target_rom') || { min: 0, max: 180 };
    
    const jointAngles = this.motionHistory.map(p => 
      p.jointAngles.find(j => j.joint === targetJoint)
    ).filter(Boolean);

    if (jointAngles.length === 0) return 0;

    const angles = jointAngles.map(j => j!.angle);
    const actualMin = Math.min(...angles);
    const actualMax = Math.max(...angles);
    const actualROM = actualMax - actualMin;
    const targetRange = targetROM.max - targetROM.min;

    return Math.min(100, Math.round(actualROM / targetRange * 100));
  }

  private calculateSymmetryScore(): number {
    // 对比左右对称关节的角度差异
    const leftShoulder = this.getAverageAngle('left_shoulder');
    const rightShoulder = this.getAverageAngle('right_shoulder');
    
    if (leftShoulder === null || rightShoulder === null) return 50;

    const diff = Math.abs(leftShoulder - rightShoulder);
    return Math.max(0, 100 - diff);
  }

  private getAverageAngle(joint: string): number | null {
    const angles = this.motionHistory
      .map(p => p.jointAngles.find(j => j.joint === joint)?.angle)
      .filter((a): a is number => a !== undefined);
    
    if (angles.length === 0) return null;
    return angles.reduce((a, b) => a + b, 0) / angles.length;
  }

  private calculateSpeedScore(): number {
    if (this.standardTrajectory.length === 0) return 50;
    // 简化：与标准轨迹的速度曲线对比
    return 70;  // 占位
  }

  private generateRecommendations(rom: number, symmetry: number, stability: number): string[] {
    const recs: string[] = [];
    if (rom < 60) recs.push('活动范围不足，请尝试加大动作幅度');
    if (symmetry < 60) recs.push('左右不对称，请注意患侧与健侧的平衡');
    if (stability < 60) recs.push('身体晃动较大，请保持核心稳定');
    if (recs.length === 0) recs.push('动作质量良好，请继续保持');
    return recs;
  }

  getMotionHistory(): MotionPoint[] {
    return [...this.motionHistory];
  }

  reset(): void {
    this.motionHistory = [];
  }
}

3.3 康复训练主界面（RehabTrainingPage.ets）

代码亮点：主界面采用三栏布局------左侧AR实时画面（患者看到自己的骨骼模型叠加），中间3D动作演示区（标准动作与患者动作对比），右侧数据面板（疼痛等级、关节角度、训练进度）。悬浮导航根据疼痛等级变色：绿色（无痛）→黄色（轻度）→橙色（中度）→红色（重度），超过阈值自动暂停训练。

typescript 复制代码

// pages/RehabTrainingPage.ets
import { PainAssessmentEngine, PainAssessment } from '../utils/PainAssessmentEngine';
import { RehabMotionTracker, RehabAssessment, MotionPoint } from '../utils/RehabMotionTracker';

@Entry
@Component
struct RehabTrainingPage {
  @State painLevel: number = 0;
  @State painTrend: string = 'stable';
  @State jointAngles: Array<{ joint: string; angle: number; status: string }> = [];
  @State rehabScore: number = 0;
  @State sessionTime: number = 0;
  @State isPaused: boolean = false;
  @State isCompleted: boolean = false;
  @State targetJoint: string = 'left_shoulder';
  @State currentROM: { min: number; max: number } = { min: 0, max: 0 };
  @State recommendations: string[] = [];
  @State navColor: string = '#00FF88';

  private painEngine: PainAssessmentEngine = PainAssessmentEngine.getInstance();
  private motionTracker: RehabMotionTracker = RehabMotionTracker.getInstance();
  private timer: number = 0;
  private arSession: arEngine.ARSession | null = null;

  aboutToAppear(): void {
    this.initializeAR();
    this.startTrainingSession();
  }

  aboutToDisappear(): void {
    clearInterval(this.timer);
    if (this.arSession) {
      this.arSession.stop();
    }
  }

  private async initializeAR(): Promise<void> {
    this.arSession = AppStorage.get<arEngine.ARSession>('ar_session');
    if (!this.arSession) return;

    // 启动AR数据循环
    this.startARLoop();
  }

  private startARLoop(): void {
    const loop = () => {
      if (!this.arSession || this.isPaused) {
        requestAnimationFrame(loop);
        return;
      }

      const frame = this.arSession.acquireFrame();
      if (!frame) {
        requestAnimationFrame(loop);
        return;
      }

      // Face AR：疼痛评估
      const faceAnchors = frame.getFaceAnchors ? frame.getFaceAnchors() : [];
      if (faceAnchors.length > 0) {
        const pain = this.painEngine.assessPain(faceAnchors[0]);
        this.updatePainState(pain);
      }

      // Body AR：动作追踪
      const bodies = frame.acquireBodySkeleton ? frame.acquireBodySkeleton() : [];
      if (bodies.length > 0) {
        const motion = this.motionTracker.trackMotion(bodies[0]);
        this.updateMotionState(motion);
      }

      frame.release();
      requestAnimationFrame(loop);
    };

    requestAnimationFrame(loop);
  }

  private updatePainState(pain: PainAssessment): void {
    this.painLevel = pain.level;
    this.painTrend = pain.trend;

    // 根据疼痛等级调整导航颜色
    this.navColor = this.getPainColor(pain.level);

    // 疼痛超过阈值自动暂停
    const config = AppStorage.get<{ painThreshold: number }>('rehab_config');
    if (config && pain.level >= config.painThreshold && !this.isPaused) {
      this.isPaused = true;
      AppStorage.setOrCreate('pain_alert', {
        level: pain.level,
        message: `疼痛等级达到${pain.level}，已自动暂停训练`,
        timestamp: Date.now()
      });
    }
  }

  private updateMotionState(motion: MotionPoint): void {
    this.jointAngles = motion.jointAngles.map(j => ({
      joint: j.joint,
      angle: Math.round(j.angle),
      status: j.status
    }));

    // 更新当前ROM
    const targetAngles = motion.jointAngles.filter(j => j.joint === this.targetJoint);
    if (targetAngles.length > 0) {
      this.currentROM = {
        min: Math.min(...targetAngles.map(j => j.angle)),
        max: Math.max(...targetAngles.map(j => j.angle))
      };
    }
  }

  private startTrainingSession(): void {
    const config = AppStorage.get<{ sessionDuration: number }>('rehab_config');
    const duration = config?.sessionDuration || 300;

    this.timer = setInterval(() => {
      if (!this.isPaused) {
        this.sessionTime++;
        
        // 每30秒评估一次
        if (this.sessionTime % 30 === 0) {
          const assessment = this.motionTracker.assessRehabQuality();
          this.rehabScore = assessment.overallScore;
          this.recommendations = assessment.recommendations;
        }

        // 训练完成
        if (this.sessionTime >= duration) {
          this.isCompleted = true;
          clearInterval(this.timer);
        }
      }
    }, 1000);
  }

  private getPainColor(level: number): string {
    if (level <= 2) return '#00FF88';      // 绿色：无痛
    if (level <= 4) return '#FFD700';      // 黄色：轻度
    if (level <= 6) return '#FF9500';      // 橙色：中度
    if (level <= 8) return '#FF4444';      // 红色：重度
    return '#8B0000';                      // 深红：极重度
  }

  private getPainLabel(level: number): string {
    if (level <= 2) return '无痛';
    if (level <= 4) return '轻度疼痛';
    if (level <= 6) return '中度疼痛';
    if (level <= 8) return '重度疼痛';
    return '极重度疼痛';
  }

  private formatTime(seconds: number): string {
    const mins = Math.floor(seconds / 60);
    const secs = seconds % 60;
    return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`;
  }

  build() {
    Row({ space: 0 }) {
      // 左侧：AR实时画面（30%）
      this.buildARViewPanel()

      // 中间：3D动作演示（40%）
      this.buildMotionDemoPanel()

      // 右侧：数据面板（30%）
      this.buildDataPanel()
    }
    .width('100%')
    .height('100%')
    .backgroundColor('#0F1419')
    .expandSafeArea(
      [SafeAreaType.SYSTEM],
      [SafeAreaEdge.TOP, SafeAreaEdge.BOTTOM, SafeAreaEdge.START, SafeAreaEdge.END]
    )
  }

  @Builder
  buildARViewPanel(): void {
    Column({ space: 0 }) {
      // AR画面标题
      Row({ space: 8 }) {
        Image($r('app.media.ic_camera'))
          .width(20)
          .height(20)
          .fillColor('#FFFFFF80')
        
        Text('实时追踪')
          .fontSize(16)
          .fontColor('#FFFFFF')
          .fontWeight(FontWeight.Medium)
      }
      .width('100%')
      .height(48)
      .padding({ left: 16, right: 16 })
      .backgroundColor('rgba(255,255,255,0.05)')

      // AR画面占位（实际应接入摄像头预览）
      Column() {
        Text('AR摄像头画面')
          .fontSize(14)
          .fontColor('#FFFFFF40')
        
        // 骨骼关键点示意
        if (this.jointAngles.length > 0) {
          Text(`${this.jointAngles.length}个关节追踪中`)
            .fontSize(12)
            .fontColor('#00FF8880')
            .margin({ top: 8 })
        }
      }
      .width('100%')
      .layoutWeight(1)
      .justifyContent(FlexAlign.Center)
      .backgroundColor('rgba(0,0,0,0.3)')

      // 疼痛等级指示器
      this.buildPainIndicator()
    }
    .layoutWeight(3)
    .height('100%')
  }

  @Builder
  buildPainIndicator(): void {
    Column({ space: 8 }) {
      Text('疼痛评估')
        .fontSize(12)
        .fontColor('rgba(255,255,255,0.5)')

      // 疼痛等级条
      Row() {
        ForEach([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], (level: number) => {
          Column()
            .width('8%')
            .height(24)
            .backgroundColor(
              level <= this.painLevel ? this.getPainColor(level) : 'rgba(255,255,255,0.1)'
            )
            .borderRadius(4)
            .margin({ right: 2 })
        })
      }
      .width('100%')
      .padding({ left: 12, right: 12 })

      Row({ space: 8 }) {
        Text(`等级: ${this.painLevel}`)
          .fontSize(14)
          .fontColor(this.navColor)
          .fontWeight(FontWeight.Bold)

        Text(this.getPainLabel(this.painLevel))
          .fontSize(12)
          .fontColor('rgba(255,255,255,0.5)')

        if (this.painTrend === 'rising') {
          Text('↑ 上升')
            .fontSize(11)
            .fontColor('#FF4444')
        } else if (this.painTrend === 'falling') {
          Text('↓ 下降')
            .fontSize(11)
            .fontColor('#00FF88')
        }
      }
    }
    .width('100%')
    .padding(12)
    .backgroundColor('rgba(255,255,255,0.03)')
  }

  @Builder
  buildMotionDemoPanel(): void {
    Column({ space: 0 }) {
      // 动作演示标题
      Row({ space: 8 }) {
        Image($r('app.media.ic_motion'))
          .width(20)
          .height(20)
          .fillColor('#FFFFFF80')
        
        Text('动作演示')
          .fontSize(16)
          .fontColor('#FFFFFF')
          .fontWeight(FontWeight.Medium)
        
        Text(this.targetJoint.replace('_', ' ').toUpperCase())
          .fontSize(12)
          .fontColor('#4A90E2')
          .layoutWeight(1)
          .textAlign(TextAlign.End)
      }
      .width('100%')
      .height(48)
      .padding({ left: 16, right: 16 })
      .backgroundColor('rgba(255,255,255,0.05)')

      // 3D动作演示区（简化）
      Column() {
        // 关节角度可视化
        if (this.jointAngles.length > 0) {
          ForEach(this.jointAngles, (joint: { joint: string; angle: number; status: string }) => {
            this.buildJointAngleBar(joint)
          })
        } else {
          Text('等待动作追踪...')
            .fontSize(14)
            .fontColor('#FFFFFF40')
        }
      }
      .width('100%')
      .layoutWeight(1)
      .padding(16)
      .justifyContent(FlexAlign.Center)

      // 训练控制按钮
      this.buildTrainingControls()
    }
    .layoutWeight(4)
    .height('100%')
    .backgroundColor('rgba(255,255,255,0.01)')
  }

  @Builder
  buildJointAngleBar(joint: { joint: string; angle: number; status: string }): void {
    Column({ space: 4 }) {
      Row({ space: 8 }) {
        Text(joint.joint.replace('_', ' '))
          .fontSize(12)
          .fontColor('#FFFFFF')
          .width(80)

        // 角度条
        Row() {
          Column()
            .width(`${Math.min(100, joint.angle / 180 * 100)}%`)
            .height(8)
            .backgroundColor(
              joint.status === 'perfect' ? '#00FF88' :
              joint.status === 'good' ? '#4A90E2' :
              joint.status === 'warning' ? '#FFD700' : '#FF4444'
            )
            .borderRadius(4)
            .animation({
              duration: 300,
              curve: Curve.EaseOut
            })
        }
        .width('60%')
        .height(8)
        .backgroundColor('rgba(255,255,255,0.1)')
        .borderRadius(4)

        Text(`${joint.angle}°`)
          .fontSize(12)
          .fontColor(
            joint.status === 'perfect' ? '#00FF88' :
            joint.status === 'good' ? '#4A90E2' :
            joint.status === 'warning' ? '#FFD700' : '#FF4444'
          )
          .width(50)
          .textAlign(TextAlign.End)
      }
      .width('100%')
      .height(32)
    }
  }

  @Builder
  buildTrainingControls(): void {
    Row({ space: 16 }) {
      // 暂停/继续
      Button() {
        Image(this.isPaused ? $r('app.media.ic_play') : $r('app.media.ic_pause'))
          .width(24)
          .height(24)
          .fillColor('#FFFFFF')
      }
      .type(ButtonType.Circle)
      .backgroundColor(this.isPaused ? '#00FF88' : '#FFD700')
      .width(56)
      .height(56)
      .shadow({ radius: 12, color: this.isPaused ? '#00FF8840' : '#FFD70040' })
      .onClick(() => {
        this.isPaused = !this.isPaused;
      })

      // 停止
      Button() {
        Image($r('app.media.ic_stop'))
          .width(24)
          .height(24)
          .fillColor('#FFFFFF')
      }
      .type(ButtonType.Circle)
      .backgroundColor('#FF4444')
      .width(48)
      .height(48)
      .onClick(() => {
        this.isPaused = true;
        // 保存训练数据
      })

      // 训练进度
      Column() {
        Text(this.formatTime(this.sessionTime))
          .fontSize(24)
          .fontWeight(FontWeight.Bold)
          .fontColor('#FFFFFF')
        
        Text('训练时长')
          .fontSize(11)
          .fontColor('rgba(255,255,255,0.5)')
      }
      .layoutWeight(1)

      // 综合评分
      Column() {
        Text(`${this.rehabScore}`)
          .fontSize(24)
          .fontWeight(FontWeight.Bold)
          .fontColor(
            this.rehabScore > 80 ? '#00FF88' :
            this.rehabScore > 60 ? '#4A90E2' : '#FFD700'
          )
        
        Text('动作评分')
          .fontSize(11)
          .fontColor('rgba(255,255,255,0.5)')
      }
    }
    .width('100%')
    .height(80)
    .padding({ left: 16, right: 16 })
    .backgroundColor('rgba(255,255,255,0.05)')
  }

  @Builder
  buildDataPanel(): void {
    Column({ space: 0 }) {
      // 数据面板标题
      Row({ space: 8 }) {
        Image($r('app.media.ic_data'))
          .width(20)
          .height(20)
          .fillColor('#FFFFFF80')
        
        Text('训练数据')
          .fontSize(16)
          .fontColor('#FFFFFF')
          .fontWeight(FontWeight.Medium)
      }
      .width('100%')
      .height(48)
      .padding({ left: 16, right: 16 })
      .backgroundColor('rgba(255,255,255,0.05)')

      // ROM数据
      Column({ space: 12 }) {
        Text('活动范围 (ROM)')
          .fontSize(14)
          .fontColor('#FFFFFF')
          .alignSelf(ItemAlign.Start)

        Row({ space: 16 }) {
          Column({ space: 4 }) {
            Text(`${this.currentROM.min}°`)
              .fontSize(28)
              .fontWeight(FontWeight.Bold)
              .fontColor('#4A90E2')
            Text('最小角度')
              .fontSize(11)
              .fontColor('rgba(255,255,255,0.5)')
          }
          .layoutWeight(1)

          Column({ space: 4 }) {
            Text(`${this.currentROM.max}°`)
              .fontSize(28)
              .fontWeight(FontWeight.Bold)
              .fontColor('#00FF88')
            Text('最大角度')
              .fontSize(11)
              .fontColor('rgba(255,255,255,0.5)')
          }
          .layoutWeight(1)
        }
        .width('100%')

        // ROM进度条
        Row() {
          Column()
            .width(`${Math.min(100, (this.currentROM.max - this.currentROM.min) / 180 * 100)}%`)
            .height(8)
            .backgroundColor('#4A90E2')
            .borderRadius(4)
        }
        .width('100%')
        .height(8)
        .backgroundColor('rgba(255,255,255,0.1)')
        .borderRadius(4)
      }
      .width('100%')
      .padding(16)
      .backgroundColor('rgba(255,255,255,0.02)')

      // 改进建议
      if (this.recommendations.length > 0) {
        Column({ space: 8 }) {
          Text('改进建议')
            .fontSize(14)
            .fontColor('#FFFFFF')
            .alignSelf(ItemAlign.Start)

          ForEach(this.recommendations, (rec: string, index: number) => {
            Row({ space: 8 }) {
              Text(`${index + 1}`)
                .fontSize(12)
                .fontColor('#FFFFFF')
                .backgroundColor('#4A90E2')
                .width(20)
                .height(20)
                .borderRadius(10)
                .textAlign(TextAlign.Center)

              Text(rec)
                .fontSize(12)
                .fontColor('rgba(255,255,255,0.7)')
                .layoutWeight(1)
            }
            .width('100%')
            .padding(8)
            .backgroundColor('rgba(255,255,255,0.03)')
            .borderRadius(8)
          })
        }
        .width('100%')
        .padding(16)
      }

      // 训练完成提示
      if (this.isCompleted) {
        Column({ space: 12 }) {
          Text('🎉 训练完成')
            .fontSize(24)
            .fontWeight(FontWeight.Bold)
            .fontColor('#00FF88')

          Text(`本次评分: ${this.rehabScore}分`)
            .fontSize(16)
            .fontColor('#FFFFFF')

          Button('查看详细报告')
            .type(ButtonType.Capsule)
            .fontSize(14)
            .fontColor('#FFFFFF')
            .backgroundColor('#4A90E2')
            .width(160)
            .height(40)
            .onClick(() => {
              // 跳转报告页面
            })
        }
        .width('100%')
        .padding(24)
        .backgroundColor('rgba(0,255,136,0.05)')
      }

      List() {
        // 历史记录
      }
      .layoutWeight(1)
    }
    .layoutWeight(3)
    .height('100%')
    .backgroundColor('rgba(0,0,0,0.2)')
  }
}

四、关键技术总结

4.1 Face AR 疼痛评估体系

疼痛指标	BlendShape参数	权重	识别特征
眉头紧张	BROW_DOWN + BROW_INNER_UP	35%	眉间皱纹、八字眉
嘴部紧张	MOUTH_FROWN + MOUTH_PUCKER	25%	嘴角下拉、抿嘴
眼部紧张	EYE_SQUINT + EYE_WIDE	25%	眯眼或异常睁大
下颌紧张	JAW_FORWARD + JAW_CLENCH	15%	咬牙、下颌前伸

4.2 Body AR 康复动作评估维度

评估维度	计算方法	达标标准	权重
活动范围(ROM)	最大角度-最小角度	达到目标范围的80%	35%
对称性	左右关节角度差	差异<15°	25%
稳定性	髋部中心位移幅度	晃动<5cm	25%
速度控制	与标准轨迹对比	偏差<20%	15%

4.3 沉浸光效与疼痛状态联动

疼痛等级	导航颜色	环境光色	触发行为
0-2 无痛	#00FF88 绿色	#1E4D2B	正常训练
3-4 轻度	#FFD700 黄色	#5C3D1E	提示注意
5-6 中度	#FF9500 橙色	#5C3D1E	建议休息
7-8 重度	#FF4444 红色	#4A1E1E	自动暂停
9-10 极重度	#8B0000 深红	#4A1E1E	紧急停止

五、调试与部署建议

5.1 调试要点

疼痛阈值个性化：不同患者对疼痛的耐受度差异大，需提供个性化校准
动作标准录制：康复师需录制标准动作轨迹作为对比基准
隐私合规：面部和骨骼数据本地处理，仅上传脱敏后的评估分数
医疗认证：作为医疗器械使用时，需通过相关认证和临床试验

5.2 部署场景

场景	设备配置	功能重点
医院康复科	大屏PC + 深度摄像头	专业评估、医生远程查看
社区康复站	平板 + 普通摄像头	标准化训练、数据上报
家庭康复	手机/平板	日常训练、家属监督
远程康复	分布式多设备	医生远程指导、实时同步

六、总结与展望

本文基于 HarmonyOS 6（API 23）的 Face AR 、Body AR 、悬浮导航 与沉浸光感四大特性，完整实战了一款"智能康复训练助手"。核心创新点：

疼痛量化评估：通过Face AR的BlendShape参数，将主观疼痛描述转化为0-10的客观等级，解决"疼痛难量化"的行业痛点
动作精准追踪：通过Body AR的20+骨骼关键点，精确计算关节角度、ROM、对称性和稳定性，实现"数字化康复师"
智能安全保护：疼痛等级超过阈值自动暂停训练，颜色编码直观提示风险等级，保障患者安全
沉浸康复体验：低刺激色调+动态光效反馈，减少患者焦虑，提升训练依从性

未来扩展方向：

AI康复方案生成：基于患者AR数据，AI自动生成个性化康复计划
远程康复指导：康复师通过分布式能力远程查看患者实时AR画面，语音指导动作纠正
游戏化康复：将枯燥的康复动作转化为AR游戏任务，提升患者参与度
区块链存证：训练数据上链存证，为医疗纠纷提供不可篡改的证据链

转载自：https://blog.csdn.net/u014727709/article/details/134326176

欢迎 👍点赞✍评论⭐收藏，欢迎指正