Pooling方法总结(语音识别)

Pooling layer将变长的frame-level features转换为一个定长的向量。

1. Statistics Pooling

链接: http://danielpovey.com/files/2017_interspeech_embeddings.pdf

The default pooling method for x-vector is statistics pooling.

The statistics pooling layer calculates the mean vector µ as well as the second-order statistics as the standard deviation vector σ over frame-level features ht (t = 1, · · · , T ).

2. Attentive Statistics Pooling

链接: https://arxiv.org/pdf/1803.10963.pdf

在一段话中,往往某些帧的帧级特征比其他帧的特征更为独特重要,因此使用attention赋予每帧feature不同的权值。

其中f(.)代表非线性变换,如tanh or ReLU function。

最后将每帧特征加劝求和

3. Self-Attentive pooling

链接:https://danielpovey.com/files/2018_interspeech_xvector_attention.pdf

4. Self Multi-Head Attention pooling

论文:Multi-Resolution Multi-Head Attention in Deep Speaker Embedding | IEEE Conference Publication | IEEE Xplore

5. NetVLAD

论文:

https://arxiv.org/pdf/1902.10107.pdf

https://arxiv.org/pdf/1511.07247.pdf

更详细的解释参考:从VLAD到NetVLAD,再到NeXtVlad - 知乎

6. Learnable Dictionary Encoding (LDE)

论文:https://arxiv.org/pdf/1804.05160.pdf

we introduce two groups of learnable parameters. One is the dictionary component center, noted as µ = {µ1, µ2 · · · µc}. The other one is assigned weights, noted as w.

where the smoothing factor for each dictionary center is learnable.

7. Attentive Bilinear Pooling (ABP) - Interspeech 2020

论文:https://www.isca-speech.org/archive/Interspeech_2020/pdfs/1922.pdf

Let be the frame-level feature map captured by the hidden layer below the self-attention layer, where L and D are the number of frames and feature dimension respectively. Then the attention map can be obtained by feeding H into a 1×1 convolutional layer followed by softmax non-linear activation, where K is the number of attention heads. The 1st-order and 2nd-order attentive statistics of H, denoted by µ and , can be computed similar as crosslayer bilinear pooling, which is

where T1(x) is the operation of reshaping x into a vector, and T2(x) includes a signed square-root step and a L2- normalization step. The output of ABP is the concatenation of µ and

8. Short-time Spectral Pooling (STSP) - ICASSP 2021

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9414094&tag=1https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9414094&tag=1From a Fourier perspective, statistics pooling only exploits the DC (zero-frequency) components in the spectral domain, whereas STSP incorporates more spectral components besides the DC ones during aggregation and is able to retain richer speaker information.

  1. 将卷积层提取到的特征做STFT(Short Time Fourier Transorm),每一个channel得到一个二维频谱图。

  2. 计算averaged spectral array

  1. 计算second-order spectral statistics
  1. 将两个特征进行拼接(C is the number of channels)

9. Multi-head attentive STSP (IEEE TRANS. ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2022)

One limitation of STSP is that the brute average of the spectrograms along the temporal axis ignores the importance of individual windowed segments when computing the spectral representations. In other words, all segments in a specific spectrogram were treated with equal importance.

相关推荐
zeroporn14 天前
以玄幻小说方式打开深度学习词嵌入算法!! 使用Skip-gram来完成 Word2Vec 词嵌入(Embedding)
人工智能·深度学习·算法·自然语言处理·embedding·word2vec·skip-gram
一个处女座的程序猿15 天前
LLMs之Embedding:Qwen3 Embedding的简介、安装和使用方法、案例应用之详细攻略
llm·embedding
酌沧17 天前
Qwen3 Embedding 结构-加载-训练 看透模型设计哲学
人工智能·embedding
羊小猪~~17 天前
【NLP入门系列三】NLP文本嵌入(以Embedding和EmbeddingBag为例)
人工智能·深度学习·神经网络·自然语言处理·大模型·nlp·embedding
jieshenai18 天前
MTEB:基于 Embedding 的文本分类评估与实战解析
人工智能·分类·embedding
LeeZhao@19 天前
【狂飙AGI】第6课:前沿技术-文生图(系列2)
人工智能·自然语言处理·aigc·embedding·agi
LeeZhao@23 天前
【狂飙AGI】第4课:前沿技术-具身智能
语言模型·自然语言处理·aigc·embedding·agi
wshzd24 天前
LLM之RAG实战(五十六)| Ollama部署下载Qwen3-Embedding向量模型和Qwen3-Reranker重排模型
embedding
Python测试之道25 天前
RAG实战:基于LangChain的《肖申克的救赎》知识问答系统构建指南
langchain·embedding·rag·deepseek
AlfredZhao1 个月前
曾经风光无限的 Oracle DBA 已经落伍了吗?
ai·vector·embedding·onnx·hnsw·ivf