在MCU上跑AI推理:ESP-DL vs TFLite Micro,选哪个?怎么优化?
文章目录
- [在MCU上跑AI推理:ESP-DL vs TFLite Micro,选哪个?怎么优化?](#在MCU上跑AI推理:ESP-DL vs TFLite Micro,选哪个?怎么优化?)
-
- 一、MCU上跑AI,不是痴人说梦
- 二、两条推理路径对比
- 三、ESP-DL:开箱即用的AI推理
-
- [3.1 目标检测(ESPDet)](#3.1 目标检测(ESPDet))
- [3.2 YOLO11系列](#3.2 YOLO11系列)
- [3.3 图像分类](#3.3 图像分类)
- [四、TFLite Micro:通用模型推理](#四、TFLite Micro:通用模型推理)
- 五、模型量化:让模型在MCU上跑得动
-
- [5.1 为什么需要量化](#5.1 为什么需要量化)
- [5.2 量化流程](#5.2 量化流程)
- 六、推理性能优化
-
- [6.1 内存管理](#6.1 内存管理)
- [6.2 分辨率与帧率](#6.2 分辨率与帧率)
- [6.3 阈值调优](#6.3 阈值调优)
- 七、完整推理流程
- 参考链接
- 总结与下篇预告
一、MCU上跑AI,不是痴人说梦
"在单片机上跑神经网络?你开玩笑吧?"
2024年我听到这句话的频率很高。但到了2025年,ESP32-P4已经能在端侧跑YOLO11n目标检测,帧率稳定在15fps,功耗不到1W。
ESP-VISION提供两条AI推理路径:ESP-DL (乐鑫自研)和TFLite Micro(Google TensorFlow Lite)。选哪个?怎么用?怎么优化?这篇文章给你答案。
二、两条推理路径对比
#mermaid-svg-Q2dHkXcUIA2pCc4r{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Q2dHkXcUIA2pCc4r .error-icon{fill:#552222;}#mermaid-svg-Q2dHkXcUIA2pCc4r .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Q2dHkXcUIA2pCc4r .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .marker.cross{stroke:#333333;}#mermaid-svg-Q2dHkXcUIA2pCc4r svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Q2dHkXcUIA2pCc4r p{margin:0;}#mermaid-svg-Q2dHkXcUIA2pCc4r .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster-label text{fill:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster-label span{color:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster-label span p{background-color:transparent;}#mermaid-svg-Q2dHkXcUIA2pCc4r .label text,#mermaid-svg-Q2dHkXcUIA2pCc4r span{fill:#333;color:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .node rect,#mermaid-svg-Q2dHkXcUIA2pCc4r .node circle,#mermaid-svg-Q2dHkXcUIA2pCc4r .node ellipse,#mermaid-svg-Q2dHkXcUIA2pCc4r .node polygon,#mermaid-svg-Q2dHkXcUIA2pCc4r .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .rough-node .label text,#mermaid-svg-Q2dHkXcUIA2pCc4r .node .label text,#mermaid-svg-Q2dHkXcUIA2pCc4r .image-shape .label,#mermaid-svg-Q2dHkXcUIA2pCc4r .icon-shape .label{text-anchor:middle;}#mermaid-svg-Q2dHkXcUIA2pCc4r .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .rough-node .label,#mermaid-svg-Q2dHkXcUIA2pCc4r .node .label,#mermaid-svg-Q2dHkXcUIA2pCc4r .image-shape .label,#mermaid-svg-Q2dHkXcUIA2pCc4r .icon-shape .label{text-align:center;}#mermaid-svg-Q2dHkXcUIA2pCc4r .node.clickable{cursor:pointer;}#mermaid-svg-Q2dHkXcUIA2pCc4r .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .arrowheadPath{fill:#333333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Q2dHkXcUIA2pCc4r .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Q2dHkXcUIA2pCc4r .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Q2dHkXcUIA2pCc4r .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster text{fill:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r .cluster span{color:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Q2dHkXcUIA2pCc4r .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Q2dHkXcUIA2pCc4r rect.text{fill:none;stroke-width:0;}#mermaid-svg-Q2dHkXcUIA2pCc4r .icon-shape,#mermaid-svg-Q2dHkXcUIA2pCc4r .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Q2dHkXcUIA2pCc4r .icon-shape p,#mermaid-svg-Q2dHkXcUIA2pCc4r .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Q2dHkXcUIA2pCc4r .icon-shape .label rect,#mermaid-svg-Q2dHkXcUIA2pCc4r .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Q2dHkXcUIA2pCc4r .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Q2dHkXcUIA2pCc4r .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Q2dHkXcUIA2pCc4r :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 乐鑫生态
TensorFlow生态
你的PyTorch/TF模型
选择部署路径
ESP-DL
TFLite Micro
转换为.espdl格式
espdl.Model()
转换为.tflite格式
tflite.Model()
ESP-DL运行时
硬件加速推理
TFLite Micro运行时
通用推理
| 对比维度 | ESP-DL | TFLite Micro |
|---|---|---|
| 模型格式 | .espdl |
.tflite |
| 支持的模型 | ESPDet、YOLO11、YOLO11nPose、ImageNet | 任意TFLite模型 |
| 高级API | ✅ 目标检测/姿态估计/分类 | ❌ 仅原始输出 |
| 量化 | 自动处理 | 需手动处理 |
| 硬件加速 | 深度优化 | 通用优化 |
| 灵活性 | 中等 | 高 |
| 上手难度 | 低 | 中高 |
💡 一句话选型:如果你的任务属于目标检测/姿态估计/图像分类,用ESP-DL;如果你需要跑自定义模型,用TFLite Micro。
三、ESP-DL:开箱即用的AI推理
3.1 目标检测(ESPDet)
python
import espdl, sensor, image, time
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time=2000)
# 加载模型(路径在/sdcard或/flash)
detector = espdl.ESPDet("/sdcard/espdet_pico.espdl")
clock = time.clock()
while True:
clock.tick()
img = sensor.snapshot()
# 推理
results = detector.detect(
img,
score_threshold=0.5, # 置信度阈值
iou_threshold=0.45, # NMS IoU阈值
max_results=10 # 最多返回几个结果
)
# 绘制结果
for result in results:
x, y, w, h = result["bbox"]
img.draw_rectangle(x, y, w, h, color=(0, 255, 0))
img.draw_string(x, y-10, f"{result['label']}: {result['score']:.2f}")
print(f"FPS: {clock.fps():.1f}")
3.2 YOLO11系列
python
# YOLO11 目标检测
detector = espdl.YOLO11("/sdcard/yolo11n.espdl")
results = detector.detect(img, score_threshold=0.5, topk=10)
# YOLO11nPose 姿态估计(17个COCO关键点)
pose_detector = espdl.YOLO11nPose("/sdcard/yolo11n_pose.espdl")
results = pose_detector.detect(img, score_threshold=0.5)
for result in results:
# result["bbox"]: 检测框
# result["keypoints"]: 17个关键点 [(x,y,conf), ...]
for kp in result["keypoints"]:
if kp[2] > 0.5: # 置信度足够
img.draw_circle(kp[0], kp[1], 3, color=(255, 0, 0))
3.3 图像分类
python
classifier = espdl.ImageNetCls("/sdcard/imagenet_cls.espdl")
results = classifier.classify(img, topk=5)
for label, score in results:
print(f"{label}: {score:.2%}")
四、TFLite Micro:通用模型推理
如果你的模型不在ESP-DL支持列表中,TFLite Micro是唯一选择:
python
import tflite, sensor, image
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time=2000)
# 加载.tflite模型
model = tflite.Model("/sdcard/my_model.tflite")
# 获取输入输出信息
input_details = model.input_details()
output_details = model.output_details()
print(f"输入shape: {input_details['shape']}")
print(f"输出shape: {output_details['shape']}")
while True:
img = sensor.snapshot()
# 预处理:匹配模型输入要求
# 1. 缩放到模型输入尺寸
img_input = img.resize(input_details['shape'][1], input_details['shape'][2])
# 2. 转换为模型期望的格式
# 这一步取决于你的模型训练时的预处理方式
# 推理
model.set_input(img_input) # 设置输入
model.invoke() # 执行推理
output = model.get_output() # 获取原始输出
# 后处理:解码原始输出
# 这需要根据你的模型输出格式自行实现
# 例如:sigmoid、softmax、NMS、anchor解码等
五、模型量化:让模型在MCU上跑得动
5.1 为什么需要量化
| 模型版本 | 大小 | 推理时间 | 精度损失 |
|---|---|---|---|
| FP32 | 25MB | 无法运行 | 0% |
| INT8量化 | 6.5MB | 80ms | <1% |
| INT8+剪枝 | 3.2MB | 45ms | ~2% |
量化把浮点tensor映射到整数值,通过scale和zero_point元数据保存映射关系。模型体积缩小4倍,推理速度提升2-3倍,精度损失通常不到1%。
5.2 量化流程
bash
# ESP-DL模型的量化由ESP-DL工具链处理
# TFLite模型使用TensorFlow的量化工具
# TFLite量化示例(在PC上执行)
import tensorflow as tf
# 加载模型
converter = tf.lite.TFLiteConverter.from_saved_model("my_model")
# INT8量化
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_data_gen # 校准数据集
quantized_model = converter.convert()
# 保存量化模型
with open("my_model_quantized.tflite", "wb") as f:
f.write(quantized_model)
六、推理性能优化
6.1 内存管理
python
# ✅ 加载一次模型,跨帧复用
detector = espdl.ESPDet("/sdcard/model.espdl")
while True:
img = sensor.snapshot()
results = detector.detect(img) # 复用同一个模型实例
# 不要每帧重建模型!
# ❌ 每帧重建模型(内存泄漏+性能灾难)
while True:
img = sensor.snapshot()
detector = espdl.ESPDet("/sdcard/model.espdl") # 每帧加载!
results = detector.detect(img)
detector.deinit() # 可以释放,但何必呢
6.2 分辨率与帧率
python
# 模型输入尺寸 vs 采集分辨率的权衡
# 假设模型输入是224×224:
# 方案A:采集VGA(640×480) → 缩小到224×224
sensor.set_framesize(sensor.VGA) # 600KB
img = sensor.snapshot()
img_input = img.resize(224, 224) # 缩放开销
# 帧率:~10 fps
# 方案B:采集QVGA(320×240) → 缩小到224×224
sensor.set_framesize(sensor.QVGA) # 150KB
img = sensor.snapshot()
img_input = img.resize(224, 224) # 缩放开销小
# 帧率:~15 fps
6.3 阈值调优
python
# 在运行时调整阈值,无需重新加载模型
results = detector.detect(
img,
score_threshold=0.3, # 光线暗时降低阈值
iou_threshold=0.45,
max_results=5
)
# vs
results = detector.detect(
img,
score_threshold=0.7, # 光线好时提高阈值,减少误检
iou_threshold=0.45,
max_results=5
)
七、完整推理流程
#mermaid-svg-tAxfiaJiuhkshhBE{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-tAxfiaJiuhkshhBE .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-tAxfiaJiuhkshhBE .error-icon{fill:#552222;}#mermaid-svg-tAxfiaJiuhkshhBE .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-tAxfiaJiuhkshhBE .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-tAxfiaJiuhkshhBE .marker{fill:#333333;stroke:#333333;}#mermaid-svg-tAxfiaJiuhkshhBE .marker.cross{stroke:#333333;}#mermaid-svg-tAxfiaJiuhkshhBE svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-tAxfiaJiuhkshhBE p{margin:0;}#mermaid-svg-tAxfiaJiuhkshhBE .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-tAxfiaJiuhkshhBE .cluster-label text{fill:#333;}#mermaid-svg-tAxfiaJiuhkshhBE .cluster-label span{color:#333;}#mermaid-svg-tAxfiaJiuhkshhBE .cluster-label span p{background-color:transparent;}#mermaid-svg-tAxfiaJiuhkshhBE .label text,#mermaid-svg-tAxfiaJiuhkshhBE span{fill:#333;color:#333;}#mermaid-svg-tAxfiaJiuhkshhBE .node rect,#mermaid-svg-tAxfiaJiuhkshhBE .node circle,#mermaid-svg-tAxfiaJiuhkshhBE .node ellipse,#mermaid-svg-tAxfiaJiuhkshhBE .node polygon,#mermaid-svg-tAxfiaJiuhkshhBE .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-tAxfiaJiuhkshhBE .rough-node .label text,#mermaid-svg-tAxfiaJiuhkshhBE .node .label text,#mermaid-svg-tAxfiaJiuhkshhBE .image-shape .label,#mermaid-svg-tAxfiaJiuhkshhBE .icon-shape .label{text-anchor:middle;}#mermaid-svg-tAxfiaJiuhkshhBE .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-tAxfiaJiuhkshhBE .rough-node .label,#mermaid-svg-tAxfiaJiuhkshhBE .node .label,#mermaid-svg-tAxfiaJiuhkshhBE .image-shape .label,#mermaid-svg-tAxfiaJiuhkshhBE .icon-shape .label{text-align:center;}#mermaid-svg-tAxfiaJiuhkshhBE .node.clickable{cursor:pointer;}#mermaid-svg-tAxfiaJiuhkshhBE .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-tAxfiaJiuhkshhBE .arrowheadPath{fill:#333333;}#mermaid-svg-tAxfiaJiuhkshhBE .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-tAxfiaJiuhkshhBE .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-tAxfiaJiuhkshhBE .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-tAxfiaJiuhkshhBE .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-tAxfiaJiuhkshhBE .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-tAxfiaJiuhkshhBE .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-tAxfiaJiuhkshhBE .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-tAxfiaJiuhkshhBE .cluster text{fill:#333;}#mermaid-svg-tAxfiaJiuhkshhBE .cluster span{color:#333;}#mermaid-svg-tAxfiaJiuhkshhBE div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-tAxfiaJiuhkshhBE .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-tAxfiaJiuhkshhBE rect.text{fill:none;stroke-width:0;}#mermaid-svg-tAxfiaJiuhkshhBE .icon-shape,#mermaid-svg-tAxfiaJiuhkshhBE .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-tAxfiaJiuhkshhBE .icon-shape p,#mermaid-svg-tAxfiaJiuhkshhBE .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-tAxfiaJiuhkshhBE .icon-shape .label rect,#mermaid-svg-tAxfiaJiuhkshhBE .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-tAxfiaJiuhkshhBE .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-tAxfiaJiuhkshhBE .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-tAxfiaJiuhkshhBE :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 模型文件
.espdl / .tflite
加载到PSRAM
model = ESPDet(path)
采集帧
sensor.snapshot()
预处理
resize / 归一化 / 量化
模型推理
model.detect() / model.invoke()
后处理
NMS / softmax / 坐标映射
绘制结果
draw_rectangle / draw_cross
输出到显示/网络
参考链接
总结与下篇预告
在MCU上跑AI的核心是选对框架+量化模型+复用实例。大多数场景用ESP-DL就够了(开箱即用),自定义模型走TFLite Micro(灵活但需要自己写后处理)。记住:模型加载一次,跨帧复用;用量化模型,不要用浮点;直接采集接近模型输入尺寸的分辨率。
下篇我们进入视频编解码------H.264硬件编码和RTSP推流,这是ESP32-P4的独占功能,也是它区别于S3的核心竞争力。
作者:码农阿虎
关键词:ESP-DL、TFLite Micro、模型量化、AI推理、端侧AI、YOLO