Key Concept:
-
Embedding: learned dense, continuous, low-dimensional representations of object 【将难以表示的对象(如图片，文本等)用连续的低维度的方式表示】
-
RNN: Recurrent Neural Network -> for processing sequential data (time series data, natural language text, audio signals, images, videos, images, ...)【用于处理序列数据】
-
CNN: Convolutional Neural Network -> for processing grid data【用于处理网格数据】Grid data means neighboring points related 相邻点之间相关的
-
Attention: summarize multiple inputs, often focusing on a small, dynamic subset of the inputs【每次关注一个小地方】
-
GNNs: Graph Neural Networks -> for processing graph data【用于处理图数据】
-
Deep Generative Models -> use deep neural networks to define generative model for complex data distributions (e.g., text, audio, image, graphs, ...)【使用深度神经网络定义复杂数据分布（如文本、音频、图像、图形等）的生成模型涉及利用各种类型的生成模型】
-
Deep Learning Frameworks: PyTorch, TensorFlow, ...
-
Gradient-based parameter estimation【基于梯度的参数估计】
- Programmers specify model (e.g., implement forward pass)【通过实现前向传递来设置模型】
- When used on training data, framework collects operations and their outputs to build computation graph【在训练数据上使用时，框架收集操作及其输出以构建计算图】
- Gradient computation performed automatically from this computation graph using backpropagation【从这个计算图中自动执行梯度计算(使用反向传播算法)】
- Optimizer uses gradient to update model【优化器使用梯度来更新模型参数】
Challenge: large, complex models; limited training data
Deep Learning复习笔记内容大纲
- Feedforward neural networks
- Backpropagation and parameter optimization
- Machine learning systems
- Training techniques for deep learning models
- Recurrent neural networks
- Convolutional neural networks
- Attention and Transformers
- Deep learning for graphs
- Deep generative modelling