离线语音识别：MLP-DNN 与 CNN 双模型
本文说明 ATS308X 上 MLP-DNN/CNN 唤醒词的数据集、训练、导出、固件和显示屏操作。 The complete INT8 inference contract is kept in auditable C/H files; the firmware does not depend on an opaque runtime.
- Dataset and artifacts
The public `mini_speech_commands_8` dataset has nine classes: `other`, `yes`, `no`, `up`, `down`, `left`, `right`, `go`, `stop`.
```text
D:\data\public_kws\mini_speech_commands_8\model MLP-DNN
D:\data\public_kws\mini_speech_commands_8\cnn_model CNN
```
Each successful run writes both formats into `--out`:
| Network | Binary | Auditable source |
|---|---|---|
| MLP-DNN | `wake_model.bin`, `wake_model.json` | `wake_model_builtin.c/.h` |
| CNN | `wake_cnn_model.bin`, `wake_cnn_model.json` | `wake_cnn_model_builtin.c/.h` |
`--c-output` can additionally copy the C/H files into `application/bt_watch/src/wakeword`. The files contain weights, biases, quantization scales, threshold, UTF-8 labels, CRC and SHA metadata.
- Input, network and output
Input is 16 kHz, 16-bit, mono PCM. The shared frontend produces 13 MFCC values for 32 time steps (`32,13`) after framing, pre-emphasis and utterance CMN. Clips with fewer than 20 voiced frames are rejected.
MLP-DNN is `416 -> 48 -> 24 -> 9`, with ReLU hidden layers and softmax output.
CNN is:
```text
1x32x13 -> Conv2D(8,3x3)+ReLU -> MaxPool(2x2)
-> Conv2D(16,3x3)+ReLU -> MaxPool(2x2)
-> Flatten(384) -> Dense(32)+ReLU -> Dense(9) -> softmax
```
The result contains `class_index`, probability and `matched`. Class zero is `other`; classes 1..8 are shown as wake words.
- Reproduce training
Run from the `` root or PyCharm Terminal:
```powershell
python application/bt_watch/tools/wakeword/wake_tool.py train `
--manifest "D:\data\public_kws\mini_speech_commands_8\dataset.json" `
--out "D:\data\public_kws\mini_speech_commands_8\model" `
--epochs 100
python application/bt_watch/tools/wakeword/cnn_wake_tool.py train `
--manifest "D:\data\public_kws\mini_speech_commands_8\dataset.json" `
--out "D:\data\public_kws\mini_speech_commands_8\cnn_model" `
--epochs 100
```
The tools validate WAV paths, hashes, split labels and non-overlapping speaker/session groups, then export `.bin/.json/.c/.h`. PowerShell uses the backtick for line continuation.
Convert existing binaries:
```powershell
python application/bt_watch/tools/wakeword/model_to_c.py --model "D:\data\public_kws\mini_speech_commands_8\model\wake_model.bin" --out-dir "D:\data\public_kws\mini_speech_commands_8\model"
python application/bt_watch/tools/wakeword/cnn_wake_tool.py convert --model "D:\data\public_kws\mini_speech_commands_8\cnn_model\wake_cnn_model.bin" --out-dir "D:\data\public_kws\mini_speech_commands_8\cnn_model"
```
- Accuracy and training time
Values below come from the generated JSON files and use INT8 inference. Background FPR is the false-positive rate on `other` clips.
| Network / split | Accuracy | Recall | Macro recall | Background FPR |
|---|---:|---:|---:|---:|
| MLP-DNN / valid | 80.87% | 76.37% | 80.10% | 0.00% |
| MLP-DNN / test | 82.14% | 76.66% | 82.11% | 0.00% |
| CNN / valid | 84.58% | 79.61% | 84.29% | 0.00% |
| CNN / test | 84.60% | 80.05% | 84.64% | 0.00% |
The historical MLP run stopped after 28 epochs. The CNN run took **693.45 s (11 min 33 s)** from loading through evaluation and bin/JSON export; C/H export took about **0.005 s**. New MLP runs write `elapsed_seconds` to `wake_model.json`.
- Dual-model firmware and logs
```text
CONFIG_APP_WAKEWORD=y
CONFIG_APP_WAKEWORD_BUILTIN_MODEL=y
CONFIG_APP_WAKEWORD_CNN=y
CONFIG_APP_WAKEWORD_DUAL_MODEL=y
```
Both networks are compiled into one image and share the same MFCC/VAD path. Log prefixes identify the backend:
```text
wakeMLP-DNN ...
wakeCNN ...
```
- Serial commands
```text
app wake load load both built-in C/H models
app wake status show active model, class count and counters
app wake model cnn select CNN while microphone is off
app wake model mlp select MLP-DNN while microphone is off
app wake on calibrate and start listening
app wake off stop listening and release ADC
app wake ui open the LVGL recognition page
```
Switching while the microphone is active returns `-EBUSY`; stop first. The page uses the same `wake_runtime_select()` API as the shell.
- Display page
`app wake ui` opens the LVGL page. It displays the current `MLP-DNN/CNN`, calibration/listening state, the prompt `请说唤醒词`, and the latest `识别结果`. Touch buttons select MLP-DNN or CNN, start/stop listening, and return. A UI timer reads a runtime snapshot, so the capture thread never writes LVGL objects.
- Firmware output
All firmware remains in:
```text
D:\Master\6project\ATS3089C_to_ATS3089P\\application\bt_watch\outdir\ats3089p_dev_watch\_firmware\
```
```powershell
cd D:\Master\6project\ATS3089C_to_ATS3089P\\zephyr
python -B build.py -n -b ats3089p_dev_watch -a bt_watch
```
| File | Size | SHA256 | CRC32 |
|---|---:|---|---|
| `ats3089p_dev_watch_261023.fw` | 24,489,472 B | `35e68368be0557552256f1366fa86b3b521ac566a52e13686278427d4b73fe01` | - |
| `ats3089p_dev_watch_261023_debug.fw` | 10,056,192 B | `93d319e56ee91d587bf8f697a61d67271608e223ae9ad00e25d1914c90c104d8` | - |
| `ota_firmware.bin` | 1,484,288 B | `74863cf8874bfe614c5c9bb10d77112aec5d3e8749626808ce423a8fbf010ac8` | `0x1FAC0ACE` |
The current build uses about 99.89% PSRAM and 99.80% SHARE_RAM; check the map before adding another large model or page.