深层rnn

目录

介绍:

​编辑

实现

---

介绍:

图画的不对。O1是不能往O2去的。只是中间的白色区域块是连接的。

第二个公式表示 用的是前一个时刻第j个隐藏层和该时刻的上一个隐藏层

这里的并行像计组里面的流水线，多条指令按序发射(关于训练时间问题)

实现

用nn.lstm举例子:2层lstm

import torch
from torch import nn
from d2l import torch as d2l

batch_size, num_steps = 32, 35
train_iter, vocab = d2l.load_data_time_machine(batch_size, num_steps)

vocab_size, num_hiddens, num_layers = len(vocab), 256, 2
num_inputs = vocab_size
device = d2l.try_gpu()
lstm_layer = nn.LSTM(num_inputs, num_hiddens, num_layers)
model = d2l.RNNModel(lstm_layer, len(vocab))
model = model.to(device)

num_epochs, lr = 500, 2
d2l.train_ch8(model, train_iter, vocab, lr*1.0, num_epochs, device)

通过num_layers的值来设定隐藏层数)