Pytorch深度学习 - 学习笔记

code_weic2023-10-20 22:10

文章目录

  • Pytorch深度学习
  • [1. Pytorch加载数据初认识](#1. Pytorch加载数据初认识)
  • [2. TensorBoard](#2. TensorBoard)
  • [3. Transforms](#3. Transforms)
  • [4. torchvision中的数据集使用](#4. torchvision中的数据集使用)
  • [5. DataLoader使用](#5. DataLoader使用)
  • [6. 神经网络](#6. 神经网络)
    • [6.1 神经网络的基本骨架](#6.1 神经网络的基本骨架)
    • [6.2 卷积层](#6.2 卷积层)
    • [6.3 最大池化的使用](#6.3 最大池化的使用)
    • [6.4 非线性激活](#6.4 非线性激活)
    • [6.5 线性层及其他层](#6.5 线性层及其他层)
    • [6.6 小实战及Sequential](#6.6 小实战及Sequential)
  • [7. 损失函数与反向传播](#7. 损失函数与反向传播)
  • [8. 优化器](#8. 优化器)
  • [9. 现有网络模型的使用](#9. 现有网络模型的使用)
  • [10. 网络模型的保存与读取](#10. 网络模型的保存与读取)
  • [11. 完整的模型训练套路](#11. 完整的模型训练套路)
  • [12. 利用GPU训练](#12. 利用GPU训练)
  • [13. 完整的模型验证套路](#13. 完整的模型验证套路)

Pytorch深度学习

dir()：打开，看见包含什么

help()：说明书

python 复制代码 
import torch
print(dir(torch))
# ['AVG', 'AggregationType', 'AliasDb', ...]

print(dir(torch.tensor))
# ['__call__', '__class__', '__delattr__', '__dir__', '__doc__',...]

1. Pytorch加载数据初认识

pytorch中读取数据主要涉及到两个类Dataset和Dataloader。

Dataset可以将可以使用的数据提取出来，并且可以对数据完成编号。即提供一种方式获取数据及其对应真实的label值。

Dataloader为网络提供不同的数据形式。

Dataset

Dataset是一个抽象类。所有数据集都应该继承Dataset，所有子类都应该重写__getitem__方法，该方法获取每个数据及其对应的label。我们可以选择重写Dataset的__len__方法

  • 如何获取每一个数据及其label

  • 告诉我们总共有多少数据

下载数据集hymenoptera_data。

Dataset测试

python 复制代码 
from torch.utils.data import Dataset
from PIL import Image
import os

# 创建MyData类，继承Dataset  Dataset是一个抽象类
# 所有数据集都应该继承Dataset，所有子类都应该重写__getitem__方法，该方法获取每个数据及其对应的label
# 我们可以选择重写Dataset的__len__方法
class MyData(Dataset):
    def __init__(self,root_dir,label_dir):
        self.root_dir = root_dir
        self.label_dir = label_dir
        self.path = os.path.join(self.root_dir,self.label_dir)
        self.image_path_list = os.listdir(self.path)

    # 获取单个图片信息
    def __getitem__(self, index):
        img_name = self.image_path_list[index]
        img_item_path = os.path.join(self.root_dir,self.label_dir,img_name)
        # 读取图片
        img = Image.open(img_item_path)
        label = self.label_dir
        return img,label

    # 获取数据集长度
    def __len__(self):
        return len(self.image_path_list )

if __name__ == '__main__':
    ants_dataset_train = MyData("data/hymenoptera_data/train","ants")
    ants_dataset_train_len = ants_dataset_train.__len__()
    print(ants_dataset_train_len)
		# 124
    bees_dataset_train = MyData("data/hymenoptera_data/train", "bees")
    bees_dataset_train_len = bees_dataset_train.__len__()
    print(bees_dataset_train_len)
		# 121
    train_dataset = ants_dataset_train + bees_dataset_train
    print(train_dataset.__len__())
		# 245
    img, label = train_dataset.__getitem__(12)
    img.show()  # 展示图片
    print(label)
    # ants

2. TensorBoard

TensorBoard是一个可视化的模块，该模块功能强大，可用于深度学习网络模型训练查看模型结构和训练效果（预测结果、网络模型结构图、准确率、loss曲线、学习率、权重分布等），可以帮我们更好的了解网络模型，设计TensorBoard调用相关代码，以上结果即可保存，是整合资料、梳理模型的好帮手。

安装TensorBoard

bash 复制代码 
pip install tensorboard

首先要导入SummaryWriterl类，直接向log_dir文件夹写入事件文件，可以被Tensorboard进行解析。

add_scalar()方法

python 复制代码 
'''
参数：
  - tag：相当于title标题
  - scalar_value：需要保存的数值，对应y轴
  - global_step：步数，对应x轴
'''
writer.add_scalar()

测试add_scalar()

python 复制代码 
from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter("logs") # 指定一个文件夹，存储事件文件

# 添加标量到summary
'''
参数：
  - tag：相当于title标题
  - scalar_value：需要保存的数值，对应y轴
  - global_step：步数，对应x轴
'''
for i in range(100):
    writer.add_scalar(tag="y=2x+1",scalar_value=2*i+1,global_step=i)

writer.close()

打开生成的文件。在pycharm控制台窗口中，切换到项目目录下，使用命令

bash 复制代码 
tensorboard --logdir=事件文件夹名称

如：
tensorboard --logdir=logs

使用命令，指定端口

tensorboard --logdir=logs --port=端口号

点击链接，打开网页查看结果。

add_image()方法

python 复制代码 
'''
参数：
  - tag：标题title
  - image_tensor：图像类型 torch.Tensor、numpy.array、string/blobname
  - global_step：步数
'''
writer.add_image()

测试add_image()

因为图片接收的数据为tensor或者numpy型，所以这里我们使用numpy型。

利用numpy.array()，对PIL图片进行转换

python 复制代码 
from torch.utils.tensorboard import SummaryWriter
from PIL import Image
import numpy as np

writer = SummaryWriter("logs")

img_path = "data/练手数据集/train/ants_image/0013035.jpg"

# Image读入图片
image_PIL = Image.open(img_path)
img_array = np.array(image_PIL)

# 添加图片
'''
参数：
  - tag：标题title
  - image_tensor：图像类型 torch.Tensor、numpy.array、string/blobname
  - global_step：步数
  - dataformats：指定img_tensor值的格式
'''
writer.add_image(tag="Test",img_tensor=img_array,global_step=1,dataformats="HWC")
writer.close()

3. Transforms

Transforms是pytorch的图像处理工具包，是torchvision模块下的一个一个类的集合，可以对图像或数据进行格式变换，裁剪，缩放，旋转等，在进行深度学习项目时用途很广泛。

ToTensor

python 复制代码 
from torchvision import transforms
from PIL import Image

image_path = "data/hymenoptera_data/train/ants/0013035.jpg"
image_PIL = Image.open(image_path)
# 示例话tensor对象
to_tensor = transforms.ToTensor()
# 调用__call__方法  实现PIL图片对象转为tensor图片对象
image_tensor = to_tensor(image_PIL)
print(image_tensor)

'''
tensor([[[0.3137, 0.3137, 0.3137,  ..., 0.3176, 0.3098, 0.2980],
         [0.3176, 0.3176, 0.3176,  ..., 0.3176, 0.3098, 0.2980],
         [0.3216, 0.3216, 0.3216,  ..., 0.3137, 0.3098, 0.3020],
         ...,
         [0.3412, 0.3412, 0.3373,  ..., 0.1725, 0.3725, 0.3529],
         [0.3412, 0.3412, 0.3373,  ..., 0.3294, 0.3529, 0.3294],
         [0.3412, 0.3412, 0.3373,  ..., 0.3098, 0.3059, 0.3294]],
         ...]]])
'''

常见的transform

关注三个点：输入、输出、作用。

Compose

把几个transforms结合在一起，按顺序执行。Compose()中的参数需要是一个列表。如：

python 复制代码 
Compose([transforms参数1,transforms参数2,...])

ToPILImage

将tensor或numpy数据类型转为PIL Image类型。

Normalize

归一化。用平均值和标准差对tensor image进行归一化。

归一化计算方式：

python 复制代码 
output[channel] = (input[channel] - mean[channel]) / std[channel]

测试：

python 复制代码 
from torchvision import transforms
from PIL import Image
from torch.utils.tensorboard import SummaryWriter

# PIL读取图片
img_PIL = Image.open("data/hymenoptera_data/train/ants/9715481_b3cb4114ff.jpg")

# 将PIL图片转为tensor
to_tensor = transforms.ToTensor()
img_tensor = to_tensor(img_PIL)

# 进行归一化
n = transforms.Normalize([0.5,0.5,0.5],[0.5,0.5,0.5])
img_norm = n(img_tensor)

writer = SummaryWriter("logs")
writer.add_image(img_tensor=img_tensor,tag="original",global_step=0)
writer.add_image(img_tensor=img_norm,tag="norm",global_step=0)
writer.close()

Resize

将PIL Image重置大小为给定的尺寸。如果size只给了一个数，则我们原图最小的边才会匹配该数值，进行等比的缩放。

测试：

python 复制代码 
from torchvision import transforms
from PIL import Image
from torch.utils.tensorboard import SummaryWriter


totensor = transforms.ToTensor()
writer = SummaryWriter("logs")

image_PIL = Image.open("data/hymenoptera_data/train/ants/0013035.jpg")
image_tensor_0 = totensor(image_PIL)
writer.add_image(tag="resize",global_step=0,img_tensor=image_tensor_0)

# 定义重置大小
resize = transforms.Resize((512,512))
image_resize = resize(image_PIL)
image_tensor_1 = totensor(image_resize)
writer.add_image(tag="resize",global_step=1,img_tensor=image_tensor_1)

writer.close()

RandomCrop

随机裁剪。

python 复制代码 
RandomCrop()
'''
参数：
  - size：给定高和宽或只给定一个数值（裁剪为正方形）。
'''

测试：

python 复制代码 
from torchvision import transforms
from PIL import Image
from torch.utils.tensorboard import SummaryWriter

image_PIL = Image.open("data/hymenoptera_data/train/ants/20935278_9190345f6b.jpg")

to_tensor = transforms.ToTensor()

# 定义裁剪
trans_random = transforms.RandomCrop(256)
trans_compose = transforms.Compose([to_tensor,trans_random])

writer = SummaryWriter("logs")

for i in range(10):
    random_img = trans_compose(image_PIL)
    writer.add_image(tag="random",img_tensor=random_img,global_step=i)

writer.close()

4. torchvision中的数据集使用

数据集：

CelebA、CIFAR、Cityscapes、COCO、DatasetFolder、EMNIST、FakeData、Fashion-MNIST、Flickr、HMDB51、ImageFolder、ImageNet、Kinetics-400、KMNIST、LSUN、MNIST、Omniglot、PhotoTour、Places365、QMNIST、SBD、SBU、STL10、SVHN、UCF101、USPS、VOC

需要设定参数：

  • root：数据集的存放位置

  • train：数据集是否为训练集

  • transform： 要对数据集进行什么变化

  • target_transform：对target进行变换

  • download：是否自动下载数据集

测试：

python 复制代码 
import torchvision.datasets

train_set = torchvision.datasets.CIFAR10(root="./dataset",train=True,download=True)
test_set = torchvision.datasets.CIFAR10(root="./dataset",train=False,download=True)
print(test_set.classes)
image,target = test_set[0]
print(image)
print(test_set.classes[target])
'''
Files already downloaded and verified
Files already downloaded and verified
['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
<PIL.Image.Image image mode=RGB size=32x32 at 0x1760360A0>
cat
'''

CIFAR-10

包含了60000张32*32对彩色图片，共有10个类别，每个类别有6000张图片。50000张图片为训练集，10000张图片为测试集。

transforms和datasets结合，测试：

python 复制代码 
import torchvision
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter

to_tensor = transforms.ToTensor()

train_set = torchvision.datasets.CIFAR10(root="./dataset",transform=to_tensor,train=True,download=True)
test_set = torchvision.datasets.CIFAR10(root="./dataset",transform=to_tensor,train=False,download=True)

writer = SummaryWriter("logs")
for i in range(10):
    image_tensor,target = train_set[i]
    writer.add_image(tag="CIFAR10",global_step=i,img_tensor=image_tensor)

writer.close()

5. DataLoader使用

python 复制代码 
DataLoader(dataset, batch_size=1, shuffle=False, sampler=None,
           batch_sampler=None, num_workers=0, collate_fn=None,
           pin_memory=False, drop_last=False, timeout=0,
           worker_init_fn=None, *, prefetch_factor=2,
           persistent_workers=False)

'''
datase：自定义的数据集
batch_size：每组数据的数量
shuffle：是否打乱数据
num_workers：加载数据，采用进行的数量。0:采用一个主进程加载数据
drop_list：分组余下的数据是否舍去。如80个数据，分组大小为30，余下20数据无法成为一组，是否舍去
'''

DataLoader会将每个batch中的img和target分别打包

测试：

python 复制代码 
import torchvision
from torch.utils.data import DataLoader
from torchvision.transforms import transforms

# 准备的测试数据集
test_data = torchvision.datasets.CIFAR10("./dataset",train=False,download=True,transform=transforms.ToTensor())

# 定义dataloader
test_loader = DataLoader(test_data,shuffle=True,batch_size=4,num_workers=0,drop_last=False)

# 测试数据集中第一张图片及target
img,target = test_data[0]
print(img.shape)
print(target)

for data in test_loader:
    imgs,targets = data
    print(imgs.shape)
    print(targets)
    
'''
torch.Size([3, 32, 32])
3
torch.Size([4, 3, 32, 32])
tensor([7, 0, 8, 7])
torch.Size([4, 3, 32, 32])
tensor([2, 6, 9, 3])
'''

使用tensorboard进行展示

python 复制代码 
import torchvision
from torch.utils.data import DataLoader
from torchvision.transforms import transforms
from torch.utils.tensorboard import SummaryWriter

# 准备的测试数据集
test_data = torchvision.datasets.CIFAR10("./dataset",train=False,download=True,transform=transforms.ToTensor())

# 定义dataloader
test_loader = DataLoader(test_data,shuffle=True,batch_size=4,num_workers=0,drop_last=False)

writer = SummaryWriter("logs")
i = 0
for data in test_loader:
    imgs,targets = data
    writer.add_images(tag="DataLoader",img_tensor=imgs,global_step=i)
    i = i + 1
writer.close()

6. 神经网络

6.1 神经网络的基本骨架

Torch.NN，（Neural network）。

模块 介绍
Module Base class for all neural network modules.所有神经网络的基类。
Sequential A sequential container.
ModuleList Holds submodules in a list.
ModuleDict Holds submodules in a dictionary.
ParameterList Holds parameters in a list.
ParameterDict Holds parameters in a dictionary.

前向传播：

定义网络，继承Module类，重写forward方法。官方示例：

python 复制代码 
import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

测试：

python 复制代码 
import torch
from torch import nn

class MyModel(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def forward(self,input):
        output = input + 1
        return output

my_model = MyModel()
x = torch.tensor(1.0)
output = my_model(x)
print(output)  # tensor(2.)

6.2 卷积层

卷积层涉及参数：

  • 滑动窗口步长
  • 卷积核尺寸
  • 边缘填充
  • 卷积核个数

特征图尺寸计算：
长度： H 2 = H 1 − F H + 2 P S + 1 宽度： W 2 = W 1 − F W + 2 P S + 1 其中， W 1 、 H 1 表示输入的宽度、长度； W 2 、 H 2 表示输出特征图的宽度、长度； F 表示卷积核长和宽的大小； S 表示滑动窗口的步长； P 表示边界填充（加几圈 0 ） 长度：H_2 = \frac{H_1-F_H+2P}{S}+1 \\ 宽度：W_2 = \frac{W_1-F_W+2P}{S}+1\\ 其中，W_1、H_1表示输入的宽度、长度；W_2、H_2表示输出特征图的宽度、长度；\\ F表示卷积核长和宽的大小；S表示滑动窗口的步长；P表示边界填充（加几圈0） 长度：H2=SH1−FH+2P+1宽度：W2=SW1−FW+2P+1其中，W1、H1表示输入的宽度、长度；W2、H2表示输出特征图的宽度、长度；F表示卷积核长和宽的大小；S表示滑动窗口的步长；P表示边界填充（加几圈0）

名称 介绍
nn.Conv1d Applies a 1D convolution over an input signal composed of several input planes.
nn.Conv2d Applies a 2D convolution over an input signal composed of several input planes.
nn.Conv3d Applies a 3D convolution over an input signal composed of several input planes.
nn.ConvTranspose1d Applies a 1D transposed convolution operator over an input image composed of several input planes.
nn.ConvTranspose2d Applies a 2D transposed convolution operator over an input image composed of several input planes.
nn.ConvTranspose3d Applies a 3D transposed convolution operator over an input image composed of several input planes.
nn.LazyConv1d A torch.nn.Conv1dmodule with lazy initialization of the in_channels argument of the Conv1d that is inferred from the input.size(1).
nn.LazyConv2d A torch.nn.Conv2dmodule with lazy initialization of the in_channels argument of the Conv2d that is inferred from the input.size(1).
nn.LazyConv3d A torch.nn.Conv3dmodule with lazy initialization of the in_channels argument of the Conv3d that is inferred from the input.size(1).
nn.LazyConvTranspose1d A torch.nn.ConvTranspose1d module with lazy initialization of the in_channels argument of the ConvTranspose1d that is inferred from the input.size(1).
nn.LazyConvTranspose2d A torch.nn.ConvTranspose2dmodule with lazy initialization of the in_channels argument of the ConvTranspose2d that is inferred from the input.size(1).
nn.LazyConvTranspose3d A torch.nn.ConvTranspose3d module with lazy initialization of the in_channels argument of the ConvTranspose3d that is inferred from the input.size(1).
nn.Unfold Extracts sliding local blocks from a batched input tensor.
nn.Fold Combines an array of sliding local blocks into a large containing tensor.

CONV2D

需要自定义卷积核时使用：

pythnon 复制代码 
torch.nn.functional.conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1) → Tensor

参数：

测试：

python 复制代码 
"""
@Author  :shw
@Date    :2023/10/17 13:35
"""
import torch.nn.functional as F
import torch

# 定义输入
input = torch.tensor([[1,2,0,3,1],
                      [0,1,2,3,1],
                      [1,2,1,0,0],
                      [5,2,3,1,1],
                      [2,1,0,1,1]])
# 定义卷积核
kernel = torch.tensor([[1,2,1],
                       [0,1,0],
                       [2,1,0]])

# CONV2D要求输入为(minibatch,in_channels,iH,iW)格式，所以要对格式进行变换
input = torch.reshape(input,(1,1,5,5))
# CONV2D要求卷积核为(out_channels,in_channels/groups,kH,kW)格式
kernel = torch.reshape(kernel,(1,1,3,3))

output = F.conv2d(input,kernel,stride=1)
print(output)
'''
tensor([[[[10, 12, 12],
          [18, 16, 16],
          [13,  9,  3]]]])
'''

不需要自定义卷积核

语法：

python 复制代码 
torch.nn.Conv2d(in_channels: int, out_channels: int, kernel_size: Union[T, Tuple[T, T]], stride: Union[T, Tuple[T, T]] = 1, padding: Union[T, Tuple[T, T]] = 0, dilation: Union[T, Tuple[T, T]] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros')

参数：

  • in_channels (int) -- Number of channels in the input image
  • out_channels (int) -- Number of channels produced by the convolution
  • kernel_size (int or tuple) -- Size of the convolving kernel
  • stride (int or tuple, optional) -- Stride of the convolution. Default: 1
  • padding (int or tuple , optional) -- Zero-padding added to both sides of the input. Default: 0
  • padding_mode (string *,* optional ) -- 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'
  • dilation (int or tuple , optional) -- Spacing between kernel elements. Default: 1
  • groups (int, optional) -- Number of blocked connections from input channels to output channels. Default: 1
  • bias (bool, optional ) -- If True, adds a learnable bias to the output. Default: True

测试：

python 复制代码 
from torchvision import datasets
from torch.nn import Conv2d
import torch
from torchvision import transforms
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

to_tensor = transforms.ToTensor()
# 读入数据
input = datasets.CIFAR10(root="./dataset",train=False,transform=to_tensor,download=True)
# 定义dataloader
dataloader = DataLoader(input,batch_size=64,shuffle=True,num_workers=0,drop_last=False)

class MyModel(torch.nn.Module):

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.conv1 = Conv2d(in_channels=3,out_channels=6,kernel_size=3,stride=1,padding=0)

    def forward(self,input):
        x = self.conv1(input)
        return x

writer = SummaryWriter("logs")

mymodel = MyModel()
step = 0
for data in dataloader:
    images,targets = data
    output = mymodel(images)
    writer.add_images("input",images,global_step=step)

    # 因为显示图片最多3个通道，这里我们输出的结果为6个通道，所以要进行变换
    # 第一个值我们不知道为多少，所以填写-1，会自动根据设定的数值进行计算
    output = torch.reshape(output,(-1,3,30,30))
    writer.add_images("output",output,global_step=step)
    step = step + 1

writer.close()

6.3 最大池化的使用

最大池化（MaxPool），也称为下采样。MaxUnpool，称为下采样。

语法：

python 复制代码 
torch.nn.MaxPool2d(kernel_size, stride=None, padding=0, dilation=1, return_indices=False, ceil_mode=False)

参数：

  • kernel_size -- the size of the window to take a max over
  • stride -- the stride of the window. Default value is kernel_size
  • padding -- implicit zero padding to be added on both sides
  • dilation -- a parameter that controls the stride of elements in the window
  • return_indices -- if True, will return the max indices along with the outputs. Useful for torch.nn.MaxUnpool2d later
  • ceil_mode -- when True, will use ceil(向上取整) instead of floor(向下取整) to compute the output shape，且保留模式。

测试：

python 复制代码 
import torch
from torch.nn import Conv2d,MaxPool2d,Module
from torch.utils.tensorboard import SummaryWriter
from torchvision import transforms

to_tensor = transforms.ToTensor()

input = torch.tensor([[1,2,0,3,1],
                      [0,1,2,3,1],
                      [1,2,1,0,0],
                      [5,2,3,1,1],
                      [2,1,0,1,1]])

input = torch.reshape(input,(-1,1,5,5))


class MyModel(Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.maxpool1 = MaxPool2d(kernel_size=3,ceil_mode=True)

    def forward(self,input):
        x = self.maxpool1(input)
        return x

mymodel = MyModel()
res = mymodel(input)
print(res)
'''
tensor([[[[2, 3],
          [5, 1]]]])
'''

通过引入数据集进行测试：

python 复制代码 
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from torch.nn import MaxPool2d,Module

to_tensor = transforms.ToTensor()

dataset = datasets.CIFAR10(root="./dataset",train=False,transform=to_tensor,download=True)
dataloader = DataLoader(dataset,batch_size=64,shuffle=True)

class MyModel(Module):

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.maxpool1 = MaxPool2d(kernel_size=3,ceil_mode=True)

    def forward(self,input):
        x = self.maxpool1(input)
        return x

writer = SummaryWriter("logs")

step = 0
mymodel = MyModel()
for data in dataloader:
    images,targets = data
    writer.add_images(tag="images",img_tensor=images,global_step=step)

    outputs = mymodel(images)
    writer.add_images(tag="maxloader",img_tensor=outputs,global_step=step)

    step = step + 1
writer.close()

6.4 非线性激活

常用的非线性激活：

  • ReLU（线性整流单元）
  • PReLU（参数线性整流单元）
  • LeakyReLU（泄漏线性整流单元）
  • ELU（指数线性单元）
  • Sigmod
  • Tanh
  • Softmax
  • ...

ReLU

测试：

python 复制代码 
from torch.nn import ReLU,Module
import torch

input = torch.tensor([[1,-0.5],
                      [-1,3]])

input = torch.reshape(input,(-1,1,2,2))

class MyModel(Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.relu1 = ReLU()

    def forward(self,input):
        output = self.relu1(input)
        return output

mymodel = MyModel()
output = mymodel(input)
print(output)
'''
tensor([[[[1., 0.],
          [0., 3.]]]])
'''

引入数据集，进行测试

python 复制代码 
"""
@Author  :shw
@Date    :2023/10/17 16:35
"""
from torch.nn import Sigmoid,Module
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

# 导入数据集
to_tensor = transforms.ToTensor()
dataset = datasets.CIFAR10(root="./dataset",train=False,transform=to_tensor,download=True)
dataloader = DataLoader(dataset,batch_size=64,shuffle=True,num_workers=0)

# 定义模型
class MyModel(Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.sigmoid1 = Sigmoid()

    def forward(self,input):
        output = self.sigmoid1(input)
        return output

writer = SummaryWriter("logs")

step = 0
mymodel = MyModel()
for data in dataloader:
    imgs,targets = data
    outputs = mymodel(imgs)
    writer.add_images(tag="img",img_tensor=imgs,global_step=step)
    writer.add_images(tag="sigmoid",img_tensor=outputs,global_step=step)
    step = step + 1

writer.close()

6.5 线性层及其他层

1. Normalization层（归一化层）

语法：

python 复制代码 
torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

参数：

  • num_features -- C* from an expected input of size(N ,C ,H ,W)
  • eps -- a value added to the denominator for numerical stability. Default: 1e-5
  • momentum -- the value used for the running_mean and running_var computation. Can be set to None for cumulative moving average (i.e. simple average). Default: 0.1
  • affine -- a boolean value that when set to True, this module has learnable affine parameters. Default: True
  • track_running_stats -- a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics, and initializes statistics buffers running_mean and running_var as None. When these buffers are None, this module always uses batch statistics. in both training and eval modes. Default: True

例子：

python 复制代码 
# With Learnable Parameters
m = nn.BatchNorm2d(100)
# Without Learnable Parameters
m = nn.BatchNorm2d(100, affine=False)
input = torch.randn(20, 100, 35, 45)
output = m(input)

2. Recurrent层（循环层）

nn.RNNBase
nn.RNN Applies a multi-layer Elman RNN with tanh⁡tanh or ReLUReLU non-linearity to an input sequence.
nn.LSTM Applies a multi-layer long short-term memory (LSTM) RNN to an input sequence.
nn.GRU Applies a multi-layer gated recurrent unit (GRU) RNN to an input sequence.
nn.RNNCell An Elman RNN cell with tanh or ReLU non-linearity.
nn.LSTMCell A long short-term memory (LSTM) cell.
nn.GRUCell A gated recurrent unit (GRU) cell

3. Transformer层

nn.Transformer A transformer model.
nn.TransformerEncoder TransformerEncoder is a stack of N encoder layers
nn.TransformerDecoder TransformerDecoder is a stack of N decoder layers
nn.TransformerEncoderLayer TransformerEncoderLayer is made up of self-attn and feedforward network.
nn.TransformerDecoderLayer TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network.

4. Dropout层

为了防止过拟合

5. Liner层（线性层）

即全连接层。

nn.Identity A placeholder identity operator that is argument-insensitive.
nn.Linear Applies a linear transformation to the incoming data: y = x T A + b y=x^TA+b y=xTA+b
nn.Bilinear Applies a bilinear transformation to the incoming data: y = x 1 T A x 2 + b y=x_1^TAx_2+b y=x1TAx2+b
nn.LazyLinear A torch.nn.Linearmodule with lazy initialization.

测试：

python 复制代码 
import torch
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
from torch.nn import Module,Linear

to_tensor = transforms.ToTensor()

dataset = datasets.CIFAR10(root="./dataset",train=True,transform=to_tensor,download=True)
dataloader = DataLoader(dataset,batch_size=64,shuffle=True,num_workers=0,drop_last=True)

class MyModel(Module):

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.linear1 = Linear(in_features=196608,out_features=10)

    def forward(self,input):
        output = self.linear1(input)
        return output

mymodel = MyModel()
for data in dataloader:
    imgs,targets = data
    imgs_flatten = torch.flatten(imgs)  # 将数据展平
    output = mymodel(imgs_flatten)
    print(output)
'''
tensor([-0.3241,  0.1549, -0.3120,  0.1616, -0.0332,  0.0360,  0.0210,  0.0720,
        -0.1955,  0.0475], grad_fn=<ViewBackward0>)
tensor([-0.0349, -0.1776, -0.5392,  0.2040,  0.1981, -0.0211, -0.0833,  0.3111,
        -0.2447, -0.2244], grad_fn=<ViewBackward0>)
        ......
'''

torch.flatten

将多维数据展平（一维）

语法：

python 复制代码 
torch.flatten(input, start_dim=0, end_dim=-1) → Tensor

参数：

  • input (Tensor) -- the input tensor.
  • start_dim (int) -- the first dim to flatten
  • end_dim (int) -- the last dim to flatten

6.6 小实战及Sequential

Sequential

语法：

python 复制代码 
torch.nn.Sequential(*args)

例子：

python 复制代码 
# Example of using Sequential
model = nn.Sequential(
          nn.Conv2d(1,20,5),
          nn.ReLU(),
          nn.Conv2d(20,64,5),
          nn.ReLU()
        )

# Example of using Sequential with OrderedDict
model = nn.Sequential(OrderedDict([
          ('conv1', nn.Conv2d(1,20,5)),
          ('relu1', nn.ReLU()),
          ('conv2', nn.Conv2d(20,64,5)),
          ('relu2', nn.ReLU())
        ]))

对CIFAR10进行分类的神经网络

python 复制代码 
import torch
from torch.nn import Module,Conv2d,Linear,MaxPool2d,Sequential,Flatten

class MyModel(Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.conv1 = Conv2d(3,32,5,padding=2,stride=1)
        self.maxpool1 = MaxPool2d(2)
        self.conv2 = Conv2d(32,32,5,padding=2,stride=1)
        self.maxpool2 = MaxPool2d(2)
        self.conv3 = Conv2d(32,64,5,padding=2,stride=1)
        self.maxpool3 = MaxPool2d(2)
        self.flatten = Flatten()
        self.linear1 = Linear(1024,64)
        self.linear2 = Linear(64,10)

    def forward(self,x):
        x = self.conv1(x)
        x = self.maxpool1(x)
        x = self.conv2(x)
        x = self.maxpool2(x)
        x = self.conv3(x)
        x = self.maxpool3(x)
        x = self.flatten(x)
        x = self.linear1(x)
        x = self.linear2(x)
        return x

mymodel = MyModel()
input = torch.ones((63,3,32,32))
output = mymodel(input)
print(output.shape)
# torch.Size([63, 10])

使用Sequential进行改进

python 复制代码 
import torch
from torch.nn import Module,Conv2d,Linear,MaxPool2d,Sequential,Flatten
from torch.utils.tensorboard import SummaryWriter

class MyModel(Module):

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2,stride=1),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2,stride=1),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2,stride=1),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )

    def forward(self,x):
        x = self.model1(x)
        return x

mymodel = MyModel()

input = torch.ones((63,3,32,32))
output = mymodel(input)
print(output.shape)
# torch.Size([63, 10])

writer = SummaryWriter("logs")  # 使用SummaryWriter展示模型
writer.add_graph(mymodel,input)
writer.close()

7. 损失函数与反向传播

损失函数：

  • 计算实际输出和目标之间的差距
  • 为我们更新输出提供一定的依据（反向传播），为每个卷积核中的参数设置了一个grad，即梯度。
markdown 复制代码 
nn.L1Loss
nn.MSELoss
nn.CrossEntropyLoss
nn.CTCLoss
nn.NLLLoss
nn.PoissonNLLLoss
nn.GaussianNLLLoss
nn.KLDivLoss
nn.BCELoss
nn.BCEWithLogitsLoss
nn.MarginRankingLoss
nn.HingeEmbeddingLoss
nn.MultiLabelMarginLoss
nn.HuberLoss
nn.SmoothL1Loss
nn.SoftMarginLoss
nn.MultiLabelSoftMarginLoss
nn.CosineEmbeddingLoss
nn.MultiMarginLoss
nn.TripletMarginLoss
nn.TripletMarginWithDistanceLoss

L1Loss测试

python 复制代码 
import torch
from torch.nn import L1Loss

inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)

inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))

loss = L1Loss()
result = loss(inputs,targets)
print(result)  # tensor(0.6667)

均方误差MSELOSS

测试：

python 复制代码 
import torch
from torch.nn import MSELoss


inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)

inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))

loss_mse = MSELoss()
result_mse = loss_mse(inputs,targets)
print(result_mse)
# tensor(1.3333)

交叉熵损失函数 CROSSENTROPYLOSS

测试：

python 复制代码 
import torch
from torch.nn import CrossEntropyLoss

cross_entropy_loss = CrossEntropyLoss()
x = torch.tensor([0.1,0.2,0.3])
y = torch.tensor([1])
x = torch.reshape(x,(1,3))

result_cross_entropy_loss = cross_entropy_loss(x,y)
print(result_cross_entropy_loss)
# tensor(1.1019)

使用数据集，测试交叉熵损失函数：

python 复制代码 
from torch.nn import Linear, Flatten, Conv2d, MaxPool2d, Sequential, Module
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
from torch import nn

to_tensor = transforms.ToTensor()
dataset = datasets.CIFAR10(root="./dataset",transform=to_tensor,download=True,train=False)
dataloader = DataLoader(dataset,batch_size=64,shuffle=True,num_workers=0)

class MyModel(Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2,stride=1),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2,stride=1),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2,stride=1),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )

    def forward(self,x):
        x = self.model1(x)
        return x

loss = nn.CrossEntropyLoss()
mymodel = MyModel()
for data in dataloader:
    images,targets = data
    outputs = mymodel(images)
    loss_res = loss(outputs,targets)
    print("loss：{}".format(loss_res))
'''
loss：2.310953140258789
loss：2.3043782711029053
loss：2.318134307861328
loss：2.308051347732544
......
'''

反向传播

python 复制代码 
loss_res.backward()  # 对损失结果 进行反向传播，求梯度

8. 优化器

根据优化器，使用反向传播求出的参数的梯度，对参数进行调整，达到误差降低目的。

Algorithms 描述
Adadelta Implements Adadelta algorithm.
Adagrad Implements Adagrad algorithm.
Adam Implements Adam algorithm.
AdamW Implements AdamW algorithm.
SparseAdam Implements lazy version of Adam algorithm suitable for sparse tensors.
Adamax Implements Adamax algorithm (a variant of Adam based on infinity norm).
ASGD Implements Averaged Stochastic Gradient Descent.
LBFGS Implements L-BFGS algorithm, heavily inspired by minFunc
RMSprop Implements RMSprop algorithm.
Rprop Implements the resilient backpropagation algorithm.
SGD Implements stochastic gradient descent (optionally with momentum).

官方例子：

python 复制代码 
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)  # lr 学习率
optimizer = optim.Adam([var1, var2], lr=0.0001)

for input, target in dataset:
    optimizer.zero_grad() # 将上一次求出的梯度清零
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    optimizer.step()  # 将参数进行调整优化

梯度下降SGD

语法：

python 复制代码 
torch.optim.SGD(params, lr=<required parameter>, momentum=0, dampening=0, weight_decay=0, nesterov=False)

参数：

  • params (iterable) -- iterable of parameters to optimize or dicts defining parameter groups
  • lr (float) -- learning rate
  • momentum (float, optional) -- momentum factor (default: 0)
  • weight_decay (float , optional) -- weight decay (L2 penalty) (default: 0)
  • dampening ([float, optional) -- dampening for momentum (default: 0)
  • nesterov (bool optional) -- enables Nesterov momentum (default: False)

测试：

python 复制代码 
from torchvision import datasets,transforms
from torch import nn,optim
from torch.utils.data import DataLoader

lr = 0.01
epochs = 10

# 导入数据
to_tensor = transforms.ToTensor()
train_data = datasets.CIFAR10(root="./dataset",train=True,transform=to_tensor,download=True)
test_data = datasets.CIFAR10(root="./dataset",train=False,transform=to_tensor,download=True)
# 处理数据，分组打包
train_dataloader = DataLoader(train_data,batch_size=64,shuffle=True,num_workers=0)
test_dataloader = DataLoader(test_data,batch_size=64,shuffle=True,num_workers=0)

# 构建网络
class Net(nn.Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model1 = nn.Sequential(
            nn.Conv2d(3, 32, 5, padding=2, stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 32, 5, padding=2, stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 5, padding=2, stride=1),
            nn.MaxPool2d(2),
            nn.Flatten(),
            nn.Linear(1024, 64),
            nn.Linear(64, 10)
        )
    def forward(self,x):
        return self.model1(x)
      
# 实例化模型
net = Net()

# 定义损失函数
loss = nn.CrossEntropyLoss()

# 定义优化器
optimer = optim.SGD(net.parameters(),lr=lr)

# 训练
for epoch in range(epochs):
    # 每个epoch，总损失
    running_loss = 0.0
    for data in train_dataloader:
        images, targets = data
        optimer.zero_grad()
        outputs = net(images)
        loss_output = loss(outputs,targets)
        loss_output.backward()
        optimer.step()
        running_loss = running_loss + loss_output
    print("epoch：{}，loss：{}".format(epoch, running_loss))
'''
epoch：0，loss：1705.0546875
epoch：1，loss：1461.0814208984375
epoch：2，loss：1306.674072265625
epoch：3，loss：1216.27001953125
epoch：4，loss：1149.858154296875
'''

9. 现有网络模型的使用

分类模型：

目标检测模型：

ImageNet数据集

注意：

  • 下载ImageNet数据集，要求已经安装了Scipy模块。
  • 数据集不能公开访问了，必须自己手动去下载数据文件，然后放在root指定的路径中

语法：

python 复制代码 
torchvision.datasets.ImageNet(root: str, split: str = 'train', **kwargs: Any)

参数：

  • root (string) -- Root directory of the ImageNet Dataset.
  • split (string *,* optional ) -- The dataset split, supports train, or val.
  • transform (callable *,* optional ) -- A function/transform that takes in an PIL image and returns a transformed version. E.g, transforms.RandomCrop
  • target_transform (callable *,* optional) -- A function/transform that takes in the target and transforms it.
  • loader -- A function to load an image given its path.

测试：

python 复制代码 
train_data = datasets.ImageNet(root="../dataset",split="train",transform=to_tensor)

VGG模型

VGG模型分类VGG11、VGG13、VGG16、VGG19。

VGG16

测试：

python 复制代码 
vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)

利用现有的网络，改动其结构。

例如：

添加一个线性层，将输出out_features为1000改为10。

python 复制代码 
from torchvision.models import vgg16,VGG16_Weights
from torch import nn

vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)
# 添加一个线性层
vgg16.add_module("add_linear",nn.Linear(1000,10))
print(vgg16)

在VGG中的classifier中添加一层

python 复制代码 
from torchvision.models import vgg16,VGG16_Weights
from torch import nn

vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)
# 在classifier中添加一层
vgg16.classifier.add_module("add_linear",nn.Linear(1000,10))
print(vgg16)

不进行添加，而在原来的基础上直接进行修改

python 复制代码 
from torchvision.models import vgg16,VGG16_Weights
from torch import nn

vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)
# 不进行添加，而在原来的基础上直接进行修改
vgg16.classifier[6] = nn.Linear(4096,10)
print(vgg16)

10. 网络模型的保存与读取

方式一：使用torch.save()保存模型，使用torch.load()读取模型

保存模型结构和模型参数

  • 保存

    python 复制代码 
    import torch                                                  
from torchvision.models import vgg16,VGG16_Weights            
                                                              
vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)    
# 保存方式 - 1                                                    
torch.save(vgg16,"./model/vgg16_method1.pth")   # 模型一般保存为pth格式

  • 读取

    python 复制代码 
    import torch

# 方式一 -> 保存方式1 ， 加载模型
vgg16_method1 = torch.load("./model/vgg16_method1.pth")
print(vgg16_method1)

方式二：

保存模型参数，官方推荐！对于大模型来说，会节省存储空间。

  • 保存

    python 复制代码 
    import torch
from torchvision.models import vgg16,VGG16_Weights

vgg16 = vgg16(weights=VGG16_Weights.DEFAULT,progress=True)

# 保存方式 - 2
# 将vgg16的状态保存为一种字典格式（不保存网络模型的结构，只保存网络模型的参数）
torch.save(vgg16.state_dict(),"./model/vgg16_method2.pth")

  • 读取

    因为直接读取到的知识参数，没有网络结构，所以我们要先创建网络结构，然后通过网络结构加载保存的参数

    python 复制代码 
    import torch
from torchvision.models import vgg16

# 方式二 -> 保存方式2 ， 加载模型
vgg16_method2 = torch.load("./model/vgg16_method2.pth")
vgg16 = vgg16()
vgg16.load_state_dict(vgg16_method2)
print(vgg16)

11. 完整的模型训练套路

使用CIFAR10数据集。

model.py

python 复制代码 
import torch
from torch import nn

# 搭建网络
class Net(nn.Module):

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model = nn.Sequential(
            nn.Conv2d(3,32,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 32, kernel_size=5, padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32,64,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Flatten(),
            nn.Linear(1024,64),
            nn.Linear(64,10)
        )

    def forward(self,x):
        return self.model(x)

# 验证网络正确性
if __name__ == '__main__':
    net = Net()
    my_input = torch.ones((64,3,32,32))
    my_output = net(my_input)
    print(my_output.shape)

work_main.py

python 复制代码 
import torch
from torch.utils.data import DataLoader
from torchvision import transforms,datasets
from torch import nn
from torch.utils.tensorboard import SummaryWriter
from model import Net

# 扫描数据集次数
epochs = 10
# 学习率
# learning_rate = 0.01
learning_rate = 1e-2   # 1e-2 = 1*10^(-2) = 0.01

to_tensor = transforms.ToTensor()
# 读取数据
train_dataset = datasets.CIFAR10(root="./dataset",train=True,download=True,transform=to_tensor)
test_dataset = datasets.CIFAR10(root="./dataset",train=False,download=True,transform=to_tensor)
# 加载数据
train_dataloader = DataLoader(train_dataset,batch_size=64,shuffle=True,num_workers=0)
test_dataloader = DataLoader(test_dataset,batch_size=64,shuffle=True,num_workers=0)

# 训练数据集大小
train_data_size = len(train_dataset)
# 测试数据集大小
test_data_size = len(test_dataset)
print("训练数据集的长度为：{}".format(train_data_size))
print("测试数据集的长度为：{}".format(test_data_size))

 # 创建网络
net = Net()
# 定义损失函数
loss = nn.CrossEntropyLoss()
# 定义优化器
optimizer = torch.optim.SGD(net.parameters(),lr=learning_rate)

# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0

# 添加 tensorboard
writer = SummaryWriter("logs")

# 训练
for epoch in range(epochs):
    print("--------------------第 {} 轮训练开始--------------------".format(epoch+1))
    for data in train_dataloader:
        images, targets = data
        outputs = net(images)
        loss_output = loss(outputs,targets)
        # 优化器优化模型
        optimizer.zero_grad()
        loss_output.backward()
        optimizer.step()

        total_train_step = total_train_step + 1

        if total_train_step%100 == 0:
            print("训练次数：{}，Loss：{}".format(total_train_step, loss_output.item()))
            writer.add_scalar(tag="Train_Loss", scalar_value=loss_output.item(), global_step=total_train_step)

    # 在每轮训练之后进行测试
    # torch.no_grad() 不进行调优
    total_test_loss = 0  # 测试总损失
    with torch.no_grad():
        for data in test_dataloader:
            images,targets = data
            outputs = net(images)
            loss_output = loss(outputs,targets)
            total_test_loss = total_test_loss + loss_output
    total_test_step = total_test_step + 1
    print("第 {} 轮，整体测试集上的Loss：{}".format(epoch,total_test_loss))
    writer.add_scalar(tag="Test_Total_Loss",scalar_value=total_test_loss,global_step=total_test_step)
    
    # 保存每一轮训练好的模型
    torch.save(net,"./model/net_{}.pth".format(epoch+1))
    print("模型net_{}.pth已保存".format(epoch+1))

writer.close()

在分类问题中，需要显示正确率衡量指标，如：

python 复制代码 
import torch

outputs = torch.tensor([[0.1,0.2],
                        [0.3,0.4]])
max_index = torch.argmax(outputs,dim=1)  # 求出每行最大值的索引
input_targets = torch.tensor([0,1])
accuracy_num = (max_index==input_targets).sum()
print("准确率为：{}".format(accuracy_num/len(max_index)))
# 准确率为：0.5

将准确率加到我们上面所写的训练模型中

只需更改work_main.py

python 复制代码 
import torch
from torch.utils.data import DataLoader
from torchvision import transforms,datasets
from torch import nn
from torch.utils.tensorboard import SummaryWriter
from model import Net

# 扫描数据集次数
epochs = 10
# 学习率
# learning_rate = 0.01
learning_rate = 1e-2   # 1e-2 = 1*10^(-2) = 0.01

to_tensor = transforms.ToTensor()
# 读取数据
train_dataset = datasets.CIFAR10(root="./dataset",train=True,download=True,transform=to_tensor)
test_dataset = datasets.CIFAR10(root="./dataset",train=False,download=True,transform=to_tensor)
# 加载数据
train_dataloader = DataLoader(train_dataset,batch_size=64,shuffle=True,num_workers=0)
test_dataloader = DataLoader(test_dataset,batch_size=64,shuffle=True,num_workers=0)

# 训练数据集大小
train_data_size = len(train_dataset)
# 测试数据集大小
test_data_size = len(test_dataset)
print("训练数据集的长度为：{}".format(train_data_size))
print("测试数据集的长度为：{}".format(test_data_size))

 # 创建网络
net = Net()
# 定义损失函数
loss = nn.CrossEntropyLoss()
# 定义优化器
optimizer = torch.optim.SGD(net.parameters(),lr=learning_rate)

# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0

# 添加 tensorboard
writer = SummaryWriter("logs")

# 训练
for epoch in range(epochs):
    print("--------------------第 {} 轮训练开始--------------------".format(epoch+1))
    for data in train_dataloader:
        images, targets = data
        outputs = net(images)
        loss_output = loss(outputs,targets)
        # 优化器优化模型
        optimizer.zero_grad()
        loss_output.backward()
        optimizer.step()

        total_train_step = total_train_step + 1

        if total_train_step%100 == 0:
            print("训练次数：{}，Loss：{}".format(total_train_step, loss_output.item()))
            writer.add_scalar(tag="Train_Loss", scalar_value=loss_output.item(), global_step=total_train_step)

    # 在每轮训练之后进行测试
    # torch.no_grad() 不进行调优
    total_test_loss = 0  # 测试总损失
    total_accuracy = 0 # 整体正确的个数
    with torch.no_grad():
        for data in test_dataloader:
            images,targets = data
            outputs = net(images)
            loss_output = loss(outputs,targets)
            total_test_loss = total_test_loss + loss_output
            accuracy = (targets == torch.argmax(outputs,dim=1)).sum() # 计算正确的个数
            total_accuracy = total_accuracy + accuracy
    total_test_step = total_test_step + 1
    print("第 {} 轮，整体测试集上的Loss：{}".format(epoch,total_test_loss))
    print("第 {} 轮，测试集上，整体准确率：{}".format(epoch,total_accuracy/test_data_size))
    writer.add_scalar(tag="Test_Total_Loss",scalar_value=total_test_loss,global_step=total_test_step)
    
    # 保存每一轮训练好的模型
    torch.save(net,"./model/net_{}.pth".format(epoch+1))
    print("模型net_{}.pth已保存".format(epoch+1))

writer.close()

在有些代码中，人们在开始训练之前加上net.train()。在开始测试之前加上net.eval()。

train()，eval()，这仅仅对某些层有影响，如Dropout层、BatchNorm层等。并非全部的网络开始训练或者测试时都需要调用该方法。

12. 利用GPU训练

第一种GPU训练方式：

  • 确定网络模型
  • 准备数据（输入，标注）
  • 损失函数
  • 调用.cuda()/或调用.to(device)方法

如：

python 复制代码 
device = torch.decive("cuda")
net = net.to(device)

使用CUDA

python 复制代码 
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from torch import nn
import torch
from torchvision import transforms,datasets

class Net(nn.Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model = nn.Sequential(
            nn.Conv2d(3,32,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 32, kernel_size=5, padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32,64,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Flatten(),
            nn.Linear(1024,64),
            nn.Linear(64,10)
        )
    def forward(self,x):
        return self.model(x)

# 扫描数据集次数
epochs = 10
# 学习率
# learning_rate = 0.01
learning_rate = 1e-2   # 1e-2 = 1*10^(-2) = 0.01

to_tensor = transforms.ToTensor()
# 读取数据
train_dataset = datasets.CIFAR10(root="./dataset",train=True,download=True,transform=to_tensor)
test_dataset = datasets.CIFAR10(root="./dataset",train=False,download=True,transform=to_tensor)
# 加载数据
train_dataloader = DataLoader(train_dataset,batch_size=64,shuffle=True,num_workers=0)
test_dataloader = DataLoader(test_dataset,batch_size=64,shuffle=True,num_workers=0)

# 训练数据集大小
train_data_size = len(train_dataset)
# 测试数据集大小
test_data_size = len(test_dataset)
print("训练数据集的长度为：{}".format(train_data_size))
print("测试数据集的长度为：{}".format(test_data_size))

 # 创建网络
net = Net()

# 定义损失函数
loss = nn.CrossEntropyLoss()
# 定义优化器
optimizer = torch.optim.SGD(net.parameters(),lr=learning_rate)

#  使用GPU！！
if torch.cuda.is_available():
  net = net.cuda()   
  loss = loss.cuda()

# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0

# 添加 tensorboard
writer = SummaryWriter("logs")

# 训练
for epoch in range(epochs):
    print("--------------------第 {} 轮训练开始--------------------".format(epoch+1))
    net.train()
    for data in train_dataloader:
        images, targets = data
        #  使用GPU！！
        if torch.cuda.is_available():
          images = images.cuda()
          targets = targets.cuda()
        outputs = net(images)
        loss_output = loss(outputs,targets)
        # 优化器优化模型
        optimizer.zero_grad()
        loss_output.backward()
        optimizer.step()

        total_train_step = total_train_step + 1

        if total_train_step%100 == 0:
            print("训练次数：{}，Loss：{}".format(total_train_step, loss_output.item()))
            writer.add_scalar(tag="Train_Loss", scalar_value=loss_output.item(), global_step=total_train_step)

    # 在每轮训练之后进行测试
    net.eval()
    total_test_loss = 0  # 测试总损失
    total_accuracy = 0 # 整体正确的个数
    # torch.no_grad() 不进行调优
    with torch.no_grad():
        for data in test_dataloader:
            images,targets = data
             #  使用GPU！！
            if torch.cuda.is_available():
              images = images.cuda()
              targets = targets.cuda()
            outputs = net(images)
            loss_output = loss(outputs,targets)
            total_test_loss = total_test_loss + loss_output
            accuracy = (targets == torch.argmax(outputs,dim=1)).sum() # 计算正确的个数
            total_accuracy = total_accuracy + accuracy
    total_test_step = total_test_step + 1
    print("第 {} 轮，整体测试集上的Loss：{}".format(epoch,total_test_loss))
    print("第 {} 轮，测试集上，整体准确率：{}".format(epoch,total_accuracy/test_data_size))
    writer.add_scalar(tag="Test_Total_Loss",scalar_value=total_test_loss,global_step=total_test_step)

    # 保存每一轮训练好的模型
    torch.save(net,"./model/net_{}.pth".format(epoch+1))
    print("模型net_{}.pth已保存".format(epoch+1))

writer.close()

在MAC上，使用MPS

python 复制代码 
import torch
from torch.utils.data import DataLoader
from torchvision import transforms,datasets
from torch import nn
from torch.utils.tensorboard import SummaryWriter

class Net(nn.Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.model = nn.Sequential(
            nn.Conv2d(3,32,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 32, kernel_size=5, padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Conv2d(32,64,kernel_size=5,padding=2,stride=1),
            nn.MaxPool2d(2),
            nn.Flatten(),
            nn.Linear(1024,64),
            nn.Linear(64,10)
        )
    def forward(self,x):
        return self.model(x)

# 扫描数据集次数
epochs = 10
# 学习率
# learning_rate = 0.01
learning_rate = 1e-2   # 1e-2 = 1*10^(-2) = 0.01

to_tensor = transforms.ToTensor()
# 读取数据
train_dataset = datasets.CIFAR10(root="./dataset",train=True,download=True,transform=to_tensor)
test_dataset = datasets.CIFAR10(root="./dataset",train=False,download=True,transform=to_tensor)
# 加载数据
train_dataloader = DataLoader(train_dataset,batch_size=64,shuffle=True,num_workers=0)
test_dataloader = DataLoader(test_dataset,batch_size=64,shuffle=True,num_workers=0)

# 训练数据集大小
train_data_size = len(train_dataset)
# 测试数据集大小
test_data_size = len(test_dataset)
print("训练数据集的长度为：{}".format(train_data_size))
print("测试数据集的长度为：{}".format(test_data_size))

 # 创建网络
net = Net()
# 定义损失函数
loss = nn.CrossEntropyLoss()
# 定义优化器
optimizer = torch.optim.SGD(net.parameters(),lr=learning_rate)

# 使用GPU
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
net.to(device)
loss.to(device)

# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0

# 添加 tensorboard
writer = SummaryWriter("gpu_logs")

# 训练
for epoch in range(epochs):
    print("--------------------第 {} 轮训练开始--------------------".format(epoch+1))
    net.train()
    for data in train_dataloader:
        images, targets = data
        # 使用GPU
        images = images.to(device)
        targets = targets.to(device)
        outputs = net(images)
        loss_output = loss(outputs,targets)
        # 优化器优化模型
        optimizer.zero_grad()
        loss_output.backward()
        optimizer.step()

        total_train_step = total_train_step + 1

        if total_train_step%100 == 0:
            print("训练次数：{}，Loss：{}".format(total_train_step, loss_output.item()))
            writer.add_scalar(tag="Train_Loss", scalar_value=loss_output.item(), global_step=total_train_step)

    # 在每轮训练之后进行测试
    net.eval()
    total_test_loss = 0  # 测试总损失
    total_accuracy = 0 # 整体正确的个数
    # torch.no_grad() 不进行调优
    with torch.no_grad():
        for data in test_dataloader:
            images,targets = data
            # 使用GPU
            images = images.to(device)
            targets = targets.to(device)
            outputs = net(images)
            loss_output = loss(outputs,targets)
            total_test_loss = total_test_loss + loss_output
            accuracy = (targets == torch.argmax(outputs,dim=1)).sum() # 计算正确的个数
            total_accuracy = total_accuracy + accuracy
    total_test_step = total_test_step + 1
    print("第 {} 轮，整体测试集上的Loss：{}".format(epoch,total_test_loss))
    print("第 {} 轮，测试集上，整体准确率：{}".format(epoch,total_accuracy/test_data_size))
    writer.add_scalar(tag="Test_Total_Loss",scalar_value=total_test_loss,global_step=total_test_step)

    # 保存每一轮训练好的模型
    torch.save(net,"./model/net_{}.pth".format(epoch+1))
    print("模型net_{}.pth已保存".format(epoch+1))

writer.close()

M1 Pro 使用GPU跑20轮，用时 0:01:39.507960 ，YYDS！
cuda分配给：

  • 网络模型对象

  • 损失函数对象

  • 训练数据输入、标签，测试数据输入、标签
    mps要分配给：

  • 网络模型对象

  • 损失函数对象

  • 训练数据输入、标签，测试数据输入、标签

13. 完整的模型验证套路

利用已经训练好的模型，然后给它提供输入。

从网上下载几张图片。

python 复制代码 
import torch
from torchvision import transforms, datasets
from PIL import Image, ImageDraw, ImageFont
from model import Net
import  os

# 读取数据 为了获取标签
test_dataset = datasets.CIFAR10(root="./dataset",train=False,download=True)
classes = test_dataset.classes

# 定义图片变换
transform = transforms.Compose([
    transforms.Resize((32,32)),
    transforms.ToTensor()
])

# 读取图片
root_dir = "./images"
image_list = os.listdir("./images")
image_tensor_list = []
image_PIL_list = []
for item in image_list:
    path = os.path.join(root_dir,item)
    image_PIL = Image.open(path)
    image_tensor = transform(image_PIL)
    image_tensor = torch.reshape(image_tensor, (-1, 3, 32, 32))
    image_tensor = image_tensor.to(device)
    image_PIL_list.append(image_PIL)
    image_tensor_list.append(image_tensor)

# 读取模型  因为训练时使用的是gpu(mps) 我们测试时要映射为使用cpu进行测试
net = torch.load("./model/net_gpu_43.pth",map_location="cpu")

# 测试
num = 0
net.eval()
with torch.no_grad():
    for item in image_tensor_list:
        output = net(item)
        output_index = torch.argmax(output, dim=1)
        out_text = classes[output_index]
        
        # 图片中写入文字
        H1 = ImageDraw.Draw(image_PIL_list[num])
        myFont = ImageFont.truetype('/System/Library/Fonts/Times.ttc', size=60)
        H1.text((30,30),out_text,fill=(255,0,0),font=myFont)
        image_PIL_list[num].save(os.path.join("./output",str(num)+".jpg"))
        num = num + 1

相关推荐
Nu11PointerException8 分钟前
JAVA笔记 | ResponseBodyEmitter等异步流式接口快速学习
笔记·学习
阿_旭1 小时前
一文读懂| 自注意力与交叉注意力机制在计算机视觉中作用与基本原理
人工智能·深度学习·计算机视觉·cross-attention·self-attention
王哈哈^_^1 小时前
【数据集】【YOLO】【目标检测】交通事故识别数据集 8939 张，YOLO道路事故目标检测实战训练教程！
前端·人工智能·深度学习·yolo·目标检测·计算机视觉·pyqt
Power20246662 小时前
NLP论文速读|LongReward:基于AI反馈来提升长上下文大语言模型
人工智能·深度学习·机器学习·自然语言处理·nlp
YRr YRr3 小时前
深度学习：循环神经网络（RNN）详解
人工智能·rnn·深度学习
sp_fyf_20243 小时前
计算机前沿技术-人工智能算法-大语言模型-最新研究进展-2024-11-01
人工智能·深度学习·神经网络·算法·机器学习·语言模型·数据挖掘
红客5973 小时前
Transformer和BERT的区别
深度学习·bert·transformer
多吃轻食3 小时前
大模型微调技术 --＞ 脉络
人工智能·深度学习·神经网络·自然语言处理·embedding
charles_vaez3 小时前
开源模型应用落地-glm模型小试-glm-4-9b-chat-快速体验（一）
深度学习·语言模型·自然语言处理
@小博的博客4 小时前
C++初阶学习第十弹——深入讲解vector的迭代器失效
数据结构·c++·学习
热门推荐
01如何才能让手机厂商主动拥抱华为，接入鸿蒙系统？02【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04Macbook pro M1 安装Ubuntu教程05基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测06【经验分享】Ubuntu22.04安装微信(linux官方版)07文件或文件夹名称中有空格如何批量去除08RAG 实践- Ollama+RagFlow 部署本地知识库09Windows10安装PCL1.14.0及点云配准10【TC3xx芯片】TC3xx芯片电源管理系统PMS详解