Pooling方法总结(语音识别)

Pooling layer将变长的frame-level features转换为一个定长的向量。

1. Statistics Pooling

链接: http://danielpovey.com/files/2017_interspeech_embeddings.pdf

The default pooling method for x-vector is statistics pooling.

The statistics pooling layer calculates the mean vector µ as well as the second-order statistics as the standard deviation vector σ over frame-level features ht (t = 1, · · · , T ).

2. Attentive Statistics Pooling

链接: https://arxiv.org/pdf/1803.10963.pdf

在一段话中,往往某些帧的帧级特征比其他帧的特征更为独特重要,因此使用attention赋予每帧feature不同的权值。

其中f(.)代表非线性变换,如tanh or ReLU function。

最后将每帧特征加劝求和

3. Self-Attentive pooling

链接:https://danielpovey.com/files/2018_interspeech_xvector_attention.pdf

4. Self Multi-Head Attention pooling

论文:Multi-Resolution Multi-Head Attention in Deep Speaker Embedding | IEEE Conference Publication | IEEE Xplore

5. NetVLAD

论文:

https://arxiv.org/pdf/1902.10107.pdf

https://arxiv.org/pdf/1511.07247.pdf

更详细的解释参考:从VLAD到NetVLAD,再到NeXtVlad - 知乎

6. Learnable Dictionary Encoding (LDE)

论文:https://arxiv.org/pdf/1804.05160.pdf

we introduce two groups of learnable parameters. One is the dictionary component center, noted as µ = {µ1, µ2 · · · µc}. The other one is assigned weights, noted as w.

where the smoothing factor for each dictionary center is learnable.

7. Attentive Bilinear Pooling (ABP) - Interspeech 2020

论文:https://www.isca-speech.org/archive/Interspeech_2020/pdfs/1922.pdf

Let be the frame-level feature map captured by the hidden layer below the self-attention layer, where L and D are the number of frames and feature dimension respectively. Then the attention map can be obtained by feeding H into a 1×1 convolutional layer followed by softmax non-linear activation, where K is the number of attention heads. The 1st-order and 2nd-order attentive statistics of H, denoted by µ and , can be computed similar as crosslayer bilinear pooling, which is

where T1(x) is the operation of reshaping x into a vector, and T2(x) includes a signed square-root step and a L2- normalization step. The output of ABP is the concatenation of µ and

8. Short-time Spectral Pooling (STSP) - ICASSP 2021

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9414094&tag=1https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9414094&tag=1From a Fourier perspective, statistics pooling only exploits the DC (zero-frequency) components in the spectral domain, whereas STSP incorporates more spectral components besides the DC ones during aggregation and is able to retain richer speaker information.

  1. 将卷积层提取到的特征做STFT(Short Time Fourier Transorm),每一个channel得到一个二维频谱图。

  2. 计算averaged spectral array

  1. 计算second-order spectral statistics
  1. 将两个特征进行拼接(C is the number of channels)

9. Multi-head attentive STSP (IEEE TRANS. ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2022)

One limitation of STSP is that the brute average of the spectrograms along the temporal axis ignores the importance of individual windowed segments when computing the spectral representations. In other words, all segments in a specific spectrogram were treated with equal importance.

相关推荐
XISHI_TIANLAN2 天前
【多模态学习】Q&A3:FFN的作用?Embedding生成方法的BERT和Word2Vec?非线性引入的作用?
学习·bert·embedding
勇往直前plus2 天前
Milvus快速入门以及用 Java 操作 Milvus
java·spring boot·embedding·milvus
ZHOU_WUYI6 天前
Qwen3-Embedding-0.6B 模型结构
embedding
你是个什么橙9 天前
自然语言处理NLP:嵌入层Embedding中input_dim的计算——Tokenizer文本分词和编码
人工智能·自然语言处理·embedding
小马过河R10 天前
GPT-5原理
人工智能·gpt·深度学习·语言模型·embedding
df007df12 天前
【RAGFlow代码详解-10】文本处理和查询处理
人工智能·ocr·embedding·llama
liliangcsdn16 天前
基于llama.cpp的量化版reranker模型调用示例
人工智能·数据分析·embedding·llama·rerank
一粒马豆17 天前
chromadb使用hugging face模型时利用镜像网站下载注意事项
python·embedding·chroma·词嵌入·hugging face·词向量·chromadb
dundunmm20 天前
【论文阅读】SIMBA: single-cell embedding along with features(2)
论文阅读·人工智能·embedding·生物信息·单细胞·多组学·细胞类型识别
dundunmm21 天前
【论文阅读】SIMBA: single-cell embedding along with features(1)
论文阅读·深度学习·神经网络·embedding·生物信息·单细胞·多组学