LSTM介绍

1.LSTM介绍

长短期记忆(Long short-term memory, LSTM)是一种特殊的RNN,主要是为了解决长序列训练过程中的梯度消失和梯度爆炸问题。简单来说,就是相比普通的RNN,LSTM能够在更长的序列中有更好的表现。其结构图如下所示:

LSTM在处理序列问题上有着很好的性能优势,所以此次我们使用LSTM模型来进行回归预测模型

本次的模型采用paddlepaddle封装的LSTM模型进行建模,首先先导入必要的包

2.导入必要的包

In [1]

javascript 复制代码
import math
import numpy as np
from numpy import concatenate
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib as mpl
from sklearn import metrics
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import LabelEncoder
import paddle

3.数据集的预处理

本次数据集为2021年数学建模研究生竞赛B题中监测点A的每小时数据。观察数据集可以发现出时间外有十一个特征,我们使用除pm2.5特征外的十个特征来对pm2.5进行每小时预测建模(实际上,我们也可以使用pm2.5本身进行单变量的建模,但考虑到其余特征对pm2.5或多或少会有影响,所以此处进行多特征建模)

原始的数据集存在缺失值,我们需要对缺失的部分进行操作,同时在神经网络训练中,为了抹平各个特征之间的差异性,会进行归一化操作。

所以对数据集我们进行如下操作:

  • 数据集序列转有监督学习;
  • 数据集缺失值处理;
  • 数据集归一化操作。

3.1数据集序列转监督函数定义

In [2]

ini 复制代码
def series_to_supervised(data,n_in=1,n_out=1,dropnan=True):
    n_vars=1 if type(data) is list else data.shape[1]
    df=pd.DataFrame(data)
    cols,names=[],[]

    for i in range(n_in,0,-1):
        cols.append(df.shift(i))
        names+=[('var%d(t-%d)' % (j+1, i)) for j in range(n_vars)]

    for i in range(0,n_out):
        cols.append(df.shift(-i))
        if i==0:
            names+=[('var%d(t)' % (j+1)) for j in range(n_vars)]
        else:
            names+=[('var%d(t+%d)' % (j+1,i)) for j in range(n_vars)]
    
    agg=pd.concat(cols,axis=1)
    agg.columns=names
    
    if dropnan:
        agg.dropna(inplace=True)
    return agg

参数说明,data为数据集本身,n_in代表多少行作为一个序列,n_out为拼接多少个序列然后得到新的数据。

3.2数据集缺失值填充集归一化函数定义

In [3]

ini 复制代码
def prepare_data(filepath, n_in, n_out=1, n_vars=6):  
    #读取数据集
    dataset=pd.read_csv(filepath,engine='python')
    dataset.fillna(method='ffill', inplace=True)#缺失值填充
    #设置时间戳索引
    dataset['date'] = pd.to_datetime(dataset['date'])
    dataset.set_index("date", inplace=True)
    values = dataset.values
    #保证所有数据都是float32类型
    values = values.astype('float32')
    #变量归一化
    scaler = MinMaxScaler(feature_range=(0, 1))
    scaled = scaler.fit_transform(values)
    #将时间序列问题转化为监督学习问题
    reframed = series_to_supervised(scaled, n_in, n_out)
    #取出保留的变量
    contain_vars = []
    for i in range(n_in, 0,-1):#for i in range(1,n_in+1):
        contain_vars += [('var%d(t-%d)' % (j, i)) for j in range(1,n_vars+1)]
    data = reframed [ contain_vars + ['var1(t)'] + [('var1(t+%d)' % (j)) for j in range(1,n_out)]]  
    #print('Reframed data:',data.head())
    #修改列名
    col_names = ['Y', 'X1', 'X2', 'X3','X4', 'X5','X6','X7','X8','X9','X10','X11']
    contain_vars = []
    for i in range(n_in, 0,-1):
        contain_vars += [('%s(t-%d)' % (col_names[j], i)) for j in range(0,n_vars)]
    data.columns = contain_vars +  ['Y(t)'] + [('Y(t+%d)' % (j)) for j in range(1,n_out)]
    # print(data.head())
    return data

参数说明,变量名相同的含义相同,n_vars表示有多少个变量,这里我们采用pandas里的"ffill"方式进行缺失值的处理,然后使用归一化对数据集进行归一化操作。

4.LSTM模型的搭建

此步骤除了搭建LSTM模型外,还定义了数据流函数,使得数据集的特征能够流入模型中进行训练

In [4]

ruby 复制代码
class LSTM(paddle.nn.Layer):
    def __init__(self,input_size=6,hidden_size=50):
        super().__init__()
        self.rnn=paddle.nn.LSTM(input_size=input_size,hidden_size=hidden_size)
        self.linear=paddle.nn.Linear(hidden_size,1)
    
    def forward(self,inputs):
        y,(hidden,cell)=self.rnn(inputs)
        output=self.linear(hidden)
        output=paddle.squeeze(output)
        return output

In [5]

ini 复制代码
class MyDataset(paddle.io.Dataset):
    def __init__(self,data,n_in=1,num_features=6,num_labels=1):
        super(MyDataset,self).__init__()
        self.data=data
        self.num_features=num_features
        self.num_labels=num_labels
        data=data.values
        x=data[:,:n_in*num_features]
        self.y=data[:,n_in*num_features]
        self.x=x.reshape((data.shape[0],n_in,num_features))
        self.x=np.array(self.x,dtype='float32')
        self.y=np.array(self.y,dtype='float32')
        self.num_sample=len(x)

    def __getitem__(self,index):
        data=self.x[index]
        label=self.y[index]
        return data,label
    
    def __len__(self):
        return self.num_sample

5.模型的训练

In [6]

ini 复制代码
data= prepare_data('hour.csv', 1, n_out=1, n_vars=11)#数据读取,此数据集变量有11个
n_train_samples = int(len(data) * 0.85)#训练集占比为0.85
train_data = data[:n_train_samples]
test_data = data[n_train_samples:]

In [7]

ini 复制代码
train_dataset = MyDataset(train_data,n_in=1,num_features=11)
test_dataset = MyDataset(test_data,n_in=1,num_features=11)
train_loader = paddle.io.DataLoader(train_dataset, batch_size=10, shuffle=False, drop_last=False)

In [8]

python 复制代码
import paddle.fluid as fluid
use_gpu = True if paddle.get_device().startswith("gpu") else False
if use_gpu:
    paddle.set_device('gpu:0')

In [9]

ini 复制代码
model = paddle.Model(LSTM(input_size=11,hidden_size=50))
model.prepare(optimizer=paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters()),
              loss=paddle.nn.MSELoss(reduction='mean'))
model.fit(train_dataset,
          epochs=30,
          batch_size=20,
          shuffle=False, 
          drop_last=True,
          verbose=1)
          
vbnet 复制代码
The loss value printed in the log is the current step, and the metric is the average value of previous steps.
Epoch 1/30
step  50/825 [>.............................] - loss: 0.0086 - ETA: 2s - 4ms/step  
python 复制代码
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:77: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  return (isinstance(seq, collections.Sequence) and
bash 复制代码
step 825/825 [==============================] - loss: 6.2217e-04 - 3ms/step          
Epoch 2/30
step 825/825 [==============================] - loss: 7.0187e-04 - 3ms/step          
Epoch 3/30
step 825/825 [==============================] - loss: 7.5661e-04 - 3ms/step          
Epoch 4/30
step 825/825 [==============================] - loss: 7.9002e-04 - 3ms/step          
Epoch 5/30
step 825/825 [==============================] - loss: 8.0973e-04 - 3ms/step          
Epoch 6/30
step 825/825 [==============================] - loss: 8.2120e-04 - 3ms/step          
Epoch 7/30
step 825/825 [==============================] - loss: 8.2733e-04 - 3ms/step          
Epoch 8/30
step 825/825 [==============================] - loss: 8.2982e-04 - 3ms/step          
Epoch 9/30
step 825/825 [==============================] - loss: 8.2988e-04 - 3ms/step          
Epoch 10/30
step 825/825 [==============================] - loss: 8.2849e-04 - 3ms/step          
Epoch 11/30
step 825/825 [==============================] - loss: 8.2655e-04 - 3ms/step          
Epoch 12/30
step 825/825 [==============================] - loss: 8.2418e-04 - 3ms/step          
Epoch 13/30
step 825/825 [==============================] - loss: 8.2167e-04 - 3ms/step          
Epoch 14/30
step 825/825 [==============================] - loss: 8.1937e-04 - 3ms/step          
Epoch 15/30
step 825/825 [==============================] - loss: 8.1763e-04 - 3ms/step          
Epoch 16/30
step 825/825 [==============================] - loss: 8.1659e-04 - 3ms/step          
Epoch 17/30
step 825/825 [==============================] - loss: 8.1617e-04 - 3ms/step          
Epoch 18/30
step 825/825 [==============================] - loss: 8.1629e-04 - 3ms/step          
Epoch 19/30
step 825/825 [==============================] - loss: 8.1685e-04 - 3ms/step          
Epoch 20/30
step 825/825 [==============================] - loss: 8.1782e-04 - 3ms/step          
Epoch 21/30
step 825/825 [==============================] - loss: 8.1926e-04 - 3ms/step          
Epoch 22/30
step 825/825 [==============================] - loss: 8.2092e-04 - 3ms/step          
Epoch 23/30
step 825/825 [==============================] - loss: 8.2267e-04 - 3ms/step          
Epoch 24/30
step 825/825 [==============================] - loss: 8.2448e-04 - 3ms/step          
Epoch 25/30
step 825/825 [==============================] - loss: 8.2632e-04 - 3ms/step          
Epoch 26/30
step 825/825 [==============================] - loss: 8.2816e-04 - 3ms/step          
Epoch 27/30
step 825/825 [==============================] - loss: 8.2998e-04 - 3ms/step          
Epoch 28/30
step 825/825 [==============================] - loss: 8.3177e-04 - 3ms/step          
Epoch 29/30
step 825/825 [==============================] - loss: 8.3355e-04 - 3ms/step          
Epoch 30/30
step 825/825 [==============================] - loss: 8.3533e-04 - 3ms/step          

6.预测任务

In [10]

scss 复制代码
# 使用验证集进行验证
y_pred = model.predict(test_dataset)
plt.plot(test_dataset.y)
plt.plot(np.concatenate(y_pred))
plt.show()
arduino 复制代码
Predict begin...
step 2915/2915 [==============================] - 2ms/step          
Predict samples: 2915
python 复制代码
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2349: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  if isinstance(obj, collections.Iterator):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2366: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  return list(data) if isinstance(data, collections.MappingView) else data
arduino 复制代码
<Figure size 432x288 with 1 Axes>

代码解释

7.小结

上述实验的是多特征的预测,实际上我们也可以使用LSTM进行单变量回归预测任务。对于一些有多特征但只限训练集,需要预测的部分没有的时序问题,我们可以先进行单变量回归预测出后续的特征值然后再进行多特征LSTM建模。

相关推荐
Ice__Cai40 分钟前
Django 视图详解(View):处理请求与返回响应的核心
数据库·后端·python·django·pip
江湖十年1 小时前
Go 官方推荐的 Go 项目文件组织方式
后端·面试·go
程序员爱钓鱼2 小时前
Go语言实战:创建一个简单的 HTTP 服务器
后端·google·go
赤橙红的黄2 小时前
SpringBoot 整合 自定义MongoDB
spring boot·后端·mongodb
程序员爱钓鱼2 小时前
Go语言实战:图的邻接表表示法实现详解
javascript·后端·go
Victor3563 小时前
MySQL(169)如何理解MySQL的二级缓存?
后端
Victor3563 小时前
MySQL(170)如何使用MySQL的性能模式(Performance Schema)?
后端
paopaokaka_luck7 小时前
基于Spring Boot+Vue的吉他社团系统设计和实现(协同过滤算法)
java·vue.js·spring boot·后端·spring
Flobby5298 小时前
Go语言新手村:轻松理解变量、常量和枚举用法
开发语言·后端·golang
Warren989 小时前
Java Stream流的使用
java·开发语言·windows·spring boot·后端·python·硬件工程