模型优化系列1：分类器CenterLoss使用Pytorch实现MNIST、CIFAR10、CIFAR100分类图示

饭碗、碗碗香2024-04-05 11:45

CentLoss实现

前言

参考文章：史上最全MNIST系列（三）------Centerloss在MNIST上的Pytorch实现（可视化）

源码：Gitee或Github都有上传，保留了最优版，对最优版调整了一些参数看效果

Gitee传送门：点击这里跳转Gitee源码库

Github传送门：点击这里跳转Github源码库

--------------------无---------------情---------------分---------------割---------------线--------------------
贴心省流：最优效果见文末的v5版

v1

一开始实现的效果还行，但！是！

我忘记参数了（0.0！！）

于是，开始了漫长的尝试。。。

效果

v2

网络结构

python 复制代码 
self.hidden_layer = nn.Sequential(
    ConvLayer(1, 32, 3, 1, 1),
    ConvLayer(32, 32, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(32, 64, 3, 1, 1),
    ConvLayer(64, 64, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(64, 128, 3, 1, 1),
    ConvLayer(128, 128, 3, 1, 1),
    nn.MaxPool2d(2)
)

self.fc = nn.Sequential(
    nn.Linear(128 * 3 * 3, 12)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=512)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.9)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.9)

效果

结论

分类效果不明显

v2_2

v2基础上仅修改了网络结构

网络结构

python 复制代码 
self.hidden_layer = nn.Sequential(
    ConvLayer(1, 32, 3, 1, 1),
    ConvLayer(32, 32, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(32, 64, 3, 1, 1),
    ConvLayer(64, 64, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(64, 128, 3, 1, 1),
    ConvLayer(128, 128, 3, 1, 1),
    nn.MaxPool2d(2)
)

self.fc = nn.Sequential(
    nn.Linear(128 * 3 * 3, 2)
)

self.output_layer = nn.Sequential(
    nn.Linear(2, 10)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=512)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.9)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.9)

效果

结论

输出12-->2+10，分类效果明确，但未完全分开

v2_3

v2_2基础上仅修改了学习率

网络结构

python 复制代码 
self.hidden_layer = nn.Sequential(
    ConvLayer(1, 32, 3, 1, 1),
    ConvLayer(32, 32, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(32, 64, 3, 1, 1),
    ConvLayer(64, 64, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(64, 128, 3, 1, 1),
    ConvLayer(128, 128, 3, 1, 1),
    nn.MaxPool2d(2)
)

self.fc = nn.Sequential(
    nn.Linear(128 * 3 * 3, 2)
)

self.output_layer = nn.Sequential(
    nn.Linear(2, 10)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=512)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.8)

效果

结论

动量0.9-->0.8，分类效果不明显

v2_4

v2_2基础上仅修改了学习率

网络输出：2+10

python 复制代码 
self.hidden_layer = nn.Sequential(
    ConvLayer(1, 32, 3, 1, 1),
    ConvLayer(32, 32, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(32, 64, 3, 1, 1),
    ConvLayer(64, 64, 3, 1, 1),
    nn.MaxPool2d(2),
    ConvLayer(64, 128, 3, 1, 1),
    ConvLayer(128, 128, 3, 1, 1),
    nn.MaxPool2d(2)
)

self.fc = nn.Sequential(
    nn.Linear(128 * 3 * 3, 2)
)

self.output_layer = nn.Sequential(
    nn.Linear(2, 10)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=512)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.9)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.5, momentum=0.9)

效果

结论

center_loss学习率0.01-->0.5，分类效果不明显

v3

网络结构

python 复制代码 
self.hidden_layer = nn.Sequential(
    ConvLayer(1, 32, 5, 1, 1),
    ConvLayer(32, 32, 5, 1, 2),
    nn.MaxPool2d(2, 2),
    ConvLayer(32, 64, 5, 1, 1),
    ConvLayer(64, 64, 5, 1, 2),
    nn.MaxPool2d(2, 2),
    ConvLayer(64, 128, 5, 1, 1),
    ConvLayer(128, 128, 5, 1, 2),
    nn.MaxPool2d(2, 2)
)

self.fc = nn.Sequential(
    nn.Linear(128, 12)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=1024)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.002, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 20, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.7)

效果

v3_2

v3基础上修改学习率

网络结构

python 复制代码 
class MainNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden_layer = nn.Sequential(
            ConvLayer(1, 32, 5, 1, 1),
            ConvLayer(32, 32, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(32, 64, 5, 1, 1),
            ConvLayer(64, 64, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(64, 128, 5, 1, 1),
            ConvLayer(128, 128, 5, 1, 2),
            nn.MaxPool2d(2, 2)
        )

        self.fc = nn.Sequential(
            nn.Linear(128, 12)
        )

    # self.output_layer = nn.Sequential(
    #     nn.Linear(2, 10)
    # )

    def forward(self, _x):
        h_out = self.hidden_layer(_x)
        h_out = h_out.reshape(-1, 128)
        # feature = self.fc(h_out)
        # outs = self.output_layer(feature)
        outs = self.fc(h_out)
        return outs

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=1024)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.8)

效果

v4

网络结构

python 复制代码 
class MainNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden_layer = nn.Sequential(
            ConvLayer(1, 32, 5, 1, 1),
            ConvLayer(32, 32, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(32, 64, 5, 1, 1),
            ConvLayer(64, 64, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(64, 128, 5, 1, 1),
            ConvLayer(128, 128, 5, 1, 2),
            nn.MaxPool2d(2, 2)
        )

        self.fc = nn.Sequential(
            nn.Linear(128, 2)
        )

        self.output_layer = nn.Sequential(
            nn.Linear(2, 10)
        )

    def forward(self, _x):
        h_out = self.hidden_layer(_x)
        h_out = h_out.reshape(-1, 128)
        feature = self.fc(h_out)
        outs = self.output_layer(feature)
        return torch.cat((feature, outs), dim=1)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=1024)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.1, momentum=0.8)

效果

v4_2

v4基础上修改了学习率

网络结构

python 复制代码 
class MainNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden_layer = nn.Sequential(
            ConvLayer(1, 32, 5, 1, 1),
            ConvLayer(32, 32, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(32, 64, 5, 1, 1),
            ConvLayer(64, 64, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(64, 128, 5, 1, 1),
            ConvLayer(128, 128, 5, 1, 2),
            nn.MaxPool2d(2, 2)
        )

        self.fc = nn.Sequential(
            nn.Linear(128, 2)
        )

        self.output_layer = nn.Sequential(
            nn.Linear(2, 10)
        )

    def forward(self, _x):
        h_out = self.hidden_layer(_x)
        h_out = h_out.reshape(-1, 128)
        feature = self.fc(h_out)
        outs = self.output_layer(feature)
        return torch.cat((feature, outs), dim=1)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=1024)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.8)

效果

结论

centerloss优化器学习率0.1-->0.01，分类效果明显

更换批次

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=256)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.01, momentum=0.8)

效果

结论

批次1024-->256，计算速度加快，分类速度减慢

v4_3

v4基础上修改了学习率

网络结构

python 复制代码 
class MainNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden_layer = nn.Sequential(
            ConvLayer(1, 32, 5, 1, 1),
            ConvLayer(32, 32, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(32, 64, 5, 1, 1),
            ConvLayer(64, 64, 5, 1, 2),
            nn.MaxPool2d(2, 2),
            ConvLayer(64, 128, 5, 1, 1),
            ConvLayer(128, 128, 5, 1, 2),
            nn.MaxPool2d(2, 2)
        )

        self.fc = nn.Sequential(
            nn.Linear(128, 2)
        )

        self.output_layer = nn.Sequential(
            nn.Linear(2, 10)
        )

    def forward(self, _x):
        h_out = self.hidden_layer(_x)
        h_out = h_out.reshape(-1, 128)
        feature = self.fc(h_out)
        outs = self.output_layer(feature)
        return torch.cat((feature, outs), dim=1)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=1024)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.8)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 30, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.5, momentum=0.8)

效果

结论

centerloss优化器学习率0.1-->0.5，分类效果明显

v5

加深网络，修改centerloss计算（取消平方根），修改学习率

centerloss计算

python 复制代码 
# return lamda / 2 * torch.mean(torch.div(torch.sqrt(torch.sum(torch.pow(_x - center_exp, 2), dim=1)), count_exp))
return lamda / 2 * torch.mean(torch.div(torch.sum(torch.pow((_x - center_exp), 2), dim=1), count_exp))

网络结构

python 复制代码 
self.hidden_layer = nn.Sequential(
    # ConvLayer(1, 32, 5, 1, 2),
    ConvLayer(3, 32, 5, 1, 2),
    ConvLayer(32, 64, 5, 1, 2),
    nn.MaxPool2d(2, 2),
    ConvLayer(64, 128, 5, 1, 2),
    ConvLayer(128, 256, 5, 1, 2),
    nn.MaxPool2d(2, 2),
    ConvLayer(256, 512, 5, 1, 2),
    ConvLayer(512, 512, 5, 1, 2),
    nn.MaxPool2d(2, 2),
    ConvLayer(512, 256, 5, 1, 2),
    ConvLayer(256, 128, 5, 1, 2),
    ConvLayer(128, 64, 5, 1, 2),
    nn.MaxPool2d(2, 2)
)

self.fc = nn.Sequential(
    # nn.Linear(64, 2)
    nn.Linear(64 * 2 * 2, 2)
)

self.output_layer = nn.Sequential(
    nn.Linear(2, 10)
)

批次和学习率

python 复制代码 
data_loader = data.DataLoader(dataset=train_data, shuffle=True, batch_size=256)
# ...
net_opt = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
scheduler = torch.optim.lr_scheduler.StepLR(net_opt, 20, gamma=0.8)
c_l_opt = torch.optim.SGD(center_loss_fn.parameters(), lr=0.5)

效果

MNIST


CIFAR10


CIFAR100

相关推荐
yzx9910134 分钟前
接口协议全解析：从HTTP到gRPC，如何选择适合你的通信方案？
网络·人工智能·网络协议·flask·pygame
sali-tec9 分钟前
C# 基于halcon的视觉工作流-章34-环状测量
开发语言·图像处理·算法·计算机视觉·c#
只说证事1 小时前
2025年数字公共治理专业重点学什么内容？（详细指南）
人工智能
LeeZhao@1 小时前
【AI推理部署】Docker篇04—Docker自动构建镜像
人工智能·docker·容器
程思扬1 小时前
利用JSONCrack与cpolar提升数据可视化及跨团队协作效率
网络·人工智能·经验分享·docker·信息可视化·容器·架构
南方者1 小时前
它的 AI Agent 凭什么能擦出火花？！
人工智能·ai编程
心动啊1211 小时前
深度神经网络1——梯度问题+标签数不够问题
人工智能·神经网络·dnn
南方者1 小时前
基于Amazon Bedrock Agent 的两个服务示例的完整流程与详细内容，包含技术架构、实现细节、交互逻辑及扩展能力
人工智能·ai编程·敏捷开发
小王爱学人工智能1 小时前
OpenCV一些进阶操作
人工智能·opencv·计算机视觉
七元权1 小时前
论文阅读-Correlate and Excite
论文阅读·深度学习·注意力机制·双目深度估计
热门推荐
01GitHub 镜像站点02UV 工具安装与国内镜像源配置指南03A股预测还能更准？开源大模型Kronos带你跑通预测+回测全流程0446个Nano-banana 精选提示词，持续更新中05KGG转MP3工具|非KGM文件|解密音频06UV安装并设置国内源07conda中设置镜像地址（附所有可换的地址）08突破百度网盘的下载限速，两种方法教会你【超详细】09Spec-Kit 使用指南10保姆级教程：手把手教你用Dify实现完美多轮对话（附Chatflow和提示词）