第J2周:ResNetV2算法学习

  1. 根据教程内容编写 ResNetV2 实现代码;
  2. 了解 ResNetV2 与 ResNetV1 的区别;

** 我的环境:**

  • 语言环境:Python3.13
  • 编译器:jupyter notebook
  • 深度学习环境:Pytorch torch 2.12+cuda12.8 torchvision 0.27

在深度学习的计算机视觉领域,何恺明团队提出的 ResNet(残差网络)无疑是一个里程碑式的突破。它通过引入跳跃连接(Skip Connection)实现了残差学习,成功解决了深度神经网络中因层数增加而导致的梯度消失和网络退化问题。

然而,在最初的 ResNet(通常称为 ResNetV1)发表不到一年后,原班人马在 2016 年的论文《Identity Mappings in Deep Residual Networks》中对其结构进行了深度优化,推出了 ResNetV2。

本文将为你详细介绍 ResNetV2 的核心改进,并直观对比它与最初的 ResNet 到底有哪些不同。

一、为什么要提出 ResNetV2?

在 ResNetV1 中,残差块(Residual Block)的基本公式可以表示为:

其中, 是输入,是经历了卷积层等操作后的残差映射。

虽然这个结构极其成功,但研究人员在后续的理论分析和数千层网络的训练中发现了一个问题:残差路径并不是一条纯粹的恒等映射。

在 V1 结构中, 和 相加之后,还需要经过一个 ReLU 激活函数。 这意味着,信息在向前传播或梯度在向后传播时,每次经过残差块,都要被 ReLU 函数砍一刀(将负数截断为 0)。如果网络深达 1000 层,这种非线性的阻断会导致信号传播变得不那么顺畅,在极深的网络中容易导致优化困难。

ResNetV2 的核心动机就是:构建一条真正畅通无阻的恒等映射通道,让信息和梯度可以没有任何阻碍地从一层直接传导到任意一层。

二、核心区别

为了实现纯粹的恒等映射,ResNetV2 对残差块内部的组件顺序进行了重组。这是 V1 和 V2 最直观、最核心的区别。

1. ResNetV1 结构

在 V1 中,信息流经一个残差块的顺序是: Conv -> BN -> ReLU -> Conv -> BN -> 与输入 x 相加 (Add) -> ReLU

  • 痛点: 最后的相加操作之后还有一个 ReLU。这个 ReLU 使得跳跃连接传过来的 发生了改变,阻碍了梯度的直接回传。

2. ResNetV2 结构

在 V2 中,研究团队将 BN 和 ReLU 提到了卷积层的前面,也就是所谓的预激活。信息流经一个残差块的顺序变成了: BN -> ReLU -> Conv -> BN -> ReLU -> Conv -> 与输入 x 相加 (Add)

在 V2 中,相加操作Add之后没有任何处理,直接输出作为下一个块的输入! 公式变成了真正的:

三、代码对比

为了更直观地理解,我们可以用伪代码的形式来看看它们的差异:

ResNetV1 (经典结构):

python 复制代码
# 主干分支
out = Conv(x)
out = BN(out)
out = ReLU(out)
out = Conv(out)
out = BN(out)

# 结合与激活
out = out + x     # 相加
out = ReLU(out)   # <-- V1的关键特征:相加后进行ReLU激活

ResNetV2 (预激活结构):

python 复制代码
# 主干分支 (先BN和ReLU,再卷积)
out = BN(x)
out = ReLU(out)
out = Conv(out)
out = BN(out)
out = ReLU(out)
out = Conv(out)

# 结合
out = out + x     # <-- V2的关键特征:相加后直接输出,无后续激活

1. 设置 GPU

python 复制代码
import torch
import torch.nn as nn
from torchvision import transforms,datasets
import os,PIL,pathlib,warnings
warnings.filterwarnings("ignore")
#忽略警告信息
device =torch.device("cuda" if torch.cuda.is_available() else "cpu")
device
复制代码
device(type='cuda')

2. 数据可视化

python 复制代码
import matplotlib.pyplot as plt
from PIL import Image
image_folder='./data/2Mild/'
#获取文件夹中的所有图像文件
image_files =[f for f in os.listdir(image_folder) if f.endswith((".jpg",".png",".jpeg"))]
fig,axes=plt.subplots(3,8,figsize=(16,6))# 创建Matplotlib图像
#使用列表推导式加载和显示图像
for ax,img_file in zip(axes.flat,image_files):
    img_path= os.path.join(image_folder,img_file)
    img =Image.open(img_path)
    ax.imshow(img)
    ax.axis('off')
#显示图像
plt.tight_layout()
plt.show()

3. 导入数据

python 复制代码
data_dir = './data/'

data_transforms = transforms.Compose([
    transforms.Resize([224, 224]),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225])
])

total_data = datasets.ImageFolder(data_dir, transform=data_transforms)
print(total_data)
print(total_data.class_to_idx)
复制代码
Dataset ImageFolder
    Number of datapoints: 1661
    Root location: ./data/
    StandardTransform
Transform: Compose(
               Resize(size=[224, 224], interpolation=bilinear, max_size=None, antialias=True)
               ToTensor()
               Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
           )
{'0Normal': 0, '2Mild': 1, '4Severe': 2}

4. 划分数据集

python 复制代码
train_size = int(0.8 * len(total_data))
test_size = len(total_data) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(total_data,[train_size, test_size])
len(train_dataset), len(test_dataset)
复制代码
(1328, 333)
python 复制代码
batch_size = 4

train_dl = torch.utils.data.DataLoader(train_dataset,
                                       batch_size=batch_size,
                                       shuffle=True)
test_dl = torch.utils.data.DataLoader(test_dataset,
                                      batch_size=batch_size)
len(train_dl),len(test_dl)
复制代码
(332, 84)

5. 检查数据

python 复制代码
for X, y in test_dl:
    print("Shape of X [N, C, H, W]: ", X.shape)
    print("Shape of y: ", y.shape, y.dtype)
    break
复制代码
Shape of X [N, C, H, W]:  torch.Size([4, 3, 224, 224])
Shape of y:  torch.Size([4]) torch.int64

四、搭建 ResNetV1 与 ResNetV2 模型

1. ResNetV1

python 复制代码
class IdentityBlockV1(nn.Module):
    def __init__(self, in_channel, filters, stride=1, kernel_size=3):
        super().__init__()
        filters1, filters2, filters3 = filters

        self.conv1 = nn.Sequential(
            nn.Conv2d(in_channel, filters1, 1, stride=stride, bias=False),
            nn.BatchNorm2d(filters1),
            nn.ReLU(True)
        )
        # kernel_size=3 时 padding 应为 kernel_size//2=1,
        # 原来写 padding=kernel_size 会把特征图越卷越大
        self.conv2 = nn.Sequential(
            nn.Conv2d(filters1, filters2, kernel_size, stride=1,
                      padding=kernel_size // 2, bias=False),
            nn.BatchNorm2d(filters2),
            nn.ReLU(True)
        )
        self.conv3 = nn.Sequential(
            nn.Conv2d(filters2, filters3, 1, stride=1, bias=False),
            nn.BatchNorm2d(filters3)
        )

        # 没有投影时 x1 和 x 通道数不一致,x1 + x 会报错
        self.use_projection = stride != 1 or in_channel != filters3
        self.shortcut = nn.Sequential(
            nn.Conv2d(in_channel, filters3, 1, stride=stride, bias=False),
            nn.BatchNorm2d(filters3)
        ) if self.use_projection else nn.Identity()

        self.relu = nn.ReLU(True)

    def forward(self, x):
        x1 = self.conv3(self.conv2(self.conv1(x)))
        x = x1 + self.shortcut(x)      # 恒等捷径或 1x1 投影捷径
        x = self.relu(x)
        return x

2. ResNetV2

python 复制代码
class IdentityBlockV2(nn.Module):
    def __init__(self, in_channel, filters, stride=1):
        super().__init__()
        filters1, filters2, filters3 = filters

        # 预激活:BN+ReLU 位于对应卷积之前
        self.bn1 = nn.BatchNorm2d(in_channel)
        self.conv1 = nn.Conv2d(
            in_channel, filters1, 1, stride=stride, bias=False)

        self.bn2 = nn.BatchNorm2d(filters1)
        self.conv2 = nn.Conv2d(
            filters1, filters2, 3, stride=1, padding=1, bias=False)

        self.bn3 = nn.BatchNorm2d(filters2)
        self.conv3 = nn.Conv2d(
            filters2, filters3, 1, stride=1, bias=False)

        self.use_projection = stride != 1 or in_channel != filters3
        self.shortcut = nn.Conv2d(
            in_channel, filters3, 1, stride=stride, bias=False
        ) if self.use_projection else nn.Identity()

        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        preact = self.relu(self.bn1(x))

        out = self.conv1(preact)
        out = self.conv2(self.relu(self.bn2(out)))
        out = self.conv3(self.relu(self.bn3(out)))

        # 恒等捷径直接传递 x;投影捷径使用预激活后的特征
        identity = self.shortcut(preact) if self.use_projection else x

        # V2:相加后直接输出,不再接 ReLU
        out = out + identity
        return out

3. ResNet-50

python 复制代码
''' 构建 ResNet-50 '''
class ResNet50(nn.Module):
    def __init__(self, ConvBlock, version='v1', num_classes=3):
        super(ResNet50, self).__init__()
        if version == 'v1':
            self.stem = nn.Sequential(
                nn.Conv2d(3, 64, 7, stride=2, padding=3, bias=False),
                nn.BatchNorm2d(64),
                nn.ReLU(),
                nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
            )
        else:
            self.stem = nn.Sequential(
                nn.Conv2d(3, 64, 7, stride=2, padding=3, bias=False),
                nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
            )

        # 每个 stage:第 1 个块负责下采样/升通道,其余为恒等块
        self.conv2 = self.make_stage(ConvBlock, 64,   [64, 64, 256],    3, 1)
        self.conv3 = self.make_stage(ConvBlock, 256,  [128, 128, 512],  4, 2)
        self.conv4 = self.make_stage(ConvBlock, 512,  [256, 256, 1024], 6, 2)
        self.conv5 = self.make_stage(ConvBlock, 1024, [512, 512, 2048], 3, 2)

        # V2 是预激活结构,最后一个残差块没有收尾 ReLU,需要补 BN+ReLU
        self.postnorm = nn.Identity() if version == 'v1' else nn.Sequential(
            nn.BatchNorm2d(2048),
            nn.ReLU(inplace=True))
        self.pool = nn.AdaptiveAvgPool2d(1)   # 不依赖输入尺寸,比 AvgPool2d(7) 稳
        self.fc = nn.Linear(2048, num_classes)

    def make_stage(self, block, in_channel, filters, blocks, stride):
        layers = [block(in_channel, filters, stride)]
        for _ in range(1, blocks):
            layers.append(block(filters[2], filters, 1))
        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.stem(x)          # 原来误写成 self.conv1(x),而 stem 存在 self.stem 里
        x = self.conv2(x)
        x = self.conv3(x)
        x = self.conv4(x)
        x = self.conv5(x)
        x = self.postnorm(x)      # 原来漏掉,V2 输出少了一次 BN+ReLU
        x = self.pool(x)
        x = torch.flatten(x, start_dim=1)
        x = self.fc(x)
        return x

4. 实例化与参数量核对

python 复制代码
num_classes = len(total_data.classes)

torch.manual_seed(50)
model_v1 = ResNet50(
    IdentityBlockV1, version='v1', num_classes=num_classes).to(device)

torch.manual_seed(50)
model_v2 = ResNet50(
    IdentityBlockV2, version='v2', num_classes=num_classes).to(device)

print('ResNetV1-50 参数量:', sum(p.numel() for p in model_v1.parameters()))
print('ResNetV2-50 参数量:', sum(p.numel() for p in model_v2.parameters()))

model_v1.eval()
model_v2.eval()
with torch.no_grad():
    test_input = torch.randn(1, 3, 224, 224).to(device)
复制代码
ResNetV1-50 参数量: 23514179
ResNetV2-50 参数量: 23506499

五、训练模型

1. 编写训练函数

python 复制代码
def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    train_loss, train_acc = 0, 0

    for X, y in dataloader:
        X, y = X.to(device), y.to(device)

        pred = model(X)
        loss = loss_fn(pred, y)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        train_acc += (pred.argmax(1) == y).type(torch.float).sum().item()
        train_loss += loss.item()

    train_acc /= size
    train_loss /= num_batches
    return train_acc, train_loss

2. 编写测试函数

python 复制代码
def test(dataloader, model, loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    test_loss, test_acc = 0, 0

    with torch.no_grad():
        for X, y in dataloader:
            X, y = X.to(device), y.to(device)

            pred = model(X)
            loss = loss_fn(pred, y)

            test_acc += (pred.argmax(1) == y).type(torch.float).sum().item()
            test_loss += loss.item()

    test_acc /= size
    test_loss /= num_batches
    return test_acc, test_loss

3 训练

python 复制代码
import copy

def fit(model, model_name, epochs=10, lr=1e-4, seed=42):
    # 两次训练使用相同种子,保持批次顺序等随机环节一致
    torch.manual_seed(seed)

    optimizer = torch.optim.AdamW(model.parameters(), lr=lr)
    loss_fn = nn.CrossEntropyLoss()

    history = {
        'train_loss': [],
        'train_acc': [],
        'test_loss': [],
        'test_acc': []
    }

    best_acc = 0.0
    best_epoch = 0
    best_model = copy.deepcopy(model)

    for epoch in range(epochs):
        model.train()
        epoch_train_acc, epoch_train_loss = train(
            train_dl, model, loss_fn, optimizer)

        model.eval()
        epoch_test_acc, epoch_test_loss = test(
            test_dl, model, loss_fn)

        if epoch_test_acc > best_acc:
            best_acc = epoch_test_acc
            best_epoch = epoch + 1
            best_model = copy.deepcopy(model)

        history['train_acc'].append(epoch_train_acc)
        history['train_loss'].append(epoch_train_loss)
        history['test_acc'].append(epoch_test_acc)
        history['test_loss'].append(epoch_test_loss)

        current_lr = optimizer.state_dict()['param_groups'][0]['lr']
        template = ('{} | Epoch:{:2d}, Train_acc:{:.1f}%, '
                    'Train_loss:{:.3f}, Test_acc:{:.1f}%, '
                    'Test_loss:{:.3f}, Lr:{:.2E}')
        print(template.format(
            model_name, epoch + 1,
            epoch_train_acc * 100, epoch_train_loss,
            epoch_test_acc * 100, epoch_test_loss,
            current_lr))

    PATH = f'./{model_name}_best_model.pth'
    torch.save(best_model.state_dict(), PATH)

    print('{} Done. Best Test_acc: {:.1f}% (Epoch {})'.format(
        model_name, best_acc * 100, best_epoch))

    return history, best_model, best_acc, best_epoch, PATH
python 复制代码
epochs = 10

history_v1, best_model_v1, best_acc_v1, best_epoch_v1, PATH_V1 = fit(
    model_v1,
    model_name='resnetv1_50',
    epochs=epochs,
    lr=1e-4,
    seed=50
)
复制代码
resnetv1_50 | Epoch: 1, Train_acc:64.2%, Train_loss:0.905, Test_acc:71.8%, Test_loss:0.701, Lr:1.00E-04
resnetv1_50 | Epoch: 2, Train_acc:74.5%, Train_loss:0.698, Test_acc:76.0%, Test_loss:0.615, Lr:1.00E-04
resnetv1_50 | Epoch: 3, Train_acc:79.4%, Train_loss:0.548, Test_acc:72.7%, Test_loss:0.849, Lr:1.00E-04
resnetv1_50 | Epoch: 4, Train_acc:82.9%, Train_loss:0.479, Test_acc:80.2%, Test_loss:0.975, Lr:1.00E-04
resnetv1_50 | Epoch: 5, Train_acc:85.5%, Train_loss:0.419, Test_acc:77.8%, Test_loss:0.571, Lr:1.00E-04
resnetv1_50 | Epoch: 6, Train_acc:85.2%, Train_loss:0.412, Test_acc:83.8%, Test_loss:1.222, Lr:1.00E-04
resnetv1_50 | Epoch: 7, Train_acc:87.0%, Train_loss:0.350, Test_acc:77.2%, Test_loss:1.124, Lr:1.00E-04
resnetv1_50 | Epoch: 8, Train_acc:88.3%, Train_loss:0.325, Test_acc:87.1%, Test_loss:0.339, Lr:1.00E-04
resnetv1_50 | Epoch: 9, Train_acc:89.7%, Train_loss:0.292, Test_acc:87.4%, Test_loss:0.384, Lr:1.00E-04
resnetv1_50 | Epoch:10, Train_acc:89.8%, Train_loss:0.301, Test_acc:89.2%, Test_loss:0.294, Lr:1.00E-04
resnetv1_50 Done. Best Test_acc: 89.2% (Epoch 10)
python 复制代码
history_v2, best_model_v2, best_acc_v2, best_epoch_v2, PATH_V2 = fit(
    model_v2,
    model_name='resnetv2_50',
    epochs=epochs,
    lr=1e-4,
    seed=50
)
复制代码
resnetv2_50 | Epoch: 1, Train_acc:65.1%, Train_loss:0.853, Test_acc:78.4%, Test_loss:0.589, Lr:1.00E-04
resnetv2_50 | Epoch: 2, Train_acc:74.4%, Train_loss:0.690, Test_acc:76.9%, Test_loss:0.634, Lr:1.00E-04
resnetv2_50 | Epoch: 3, Train_acc:77.3%, Train_loss:0.610, Test_acc:77.2%, Test_loss:0.532, Lr:1.00E-04
resnetv2_50 | Epoch: 4, Train_acc:81.3%, Train_loss:0.506, Test_acc:85.6%, Test_loss:0.408, Lr:1.00E-04
resnetv2_50 | Epoch: 5, Train_acc:83.3%, Train_loss:0.476, Test_acc:80.8%, Test_loss:0.568, Lr:1.00E-04
resnetv2_50 | Epoch: 6, Train_acc:83.5%, Train_loss:0.456, Test_acc:82.3%, Test_loss:0.409, Lr:1.00E-04
resnetv2_50 | Epoch: 7, Train_acc:85.2%, Train_loss:0.399, Test_acc:86.8%, Test_loss:0.371, Lr:1.00E-04
resnetv2_50 | Epoch: 8, Train_acc:87.4%, Train_loss:0.344, Test_acc:86.2%, Test_loss:0.363, Lr:1.00E-04
resnetv2_50 | Epoch: 9, Train_acc:88.2%, Train_loss:0.319, Test_acc:83.2%, Test_loss:0.425, Lr:1.00E-04
resnetv2_50 | Epoch:10, Train_acc:88.9%, Train_loss:0.308, Test_acc:84.7%, Test_loss:0.419, Lr:1.00E-04
resnetv2_50 Done. Best Test_acc: 86.8% (Epoch 7)l六结果可视化

六、结果可视化

python 复制代码
import matplotlib.pyplot as plt
import warnings
from datetime import datetime

def plot_compare(histories, save_path='compare_v1_v2.png'):
    # histories: {图例名: fit() 返回的 history 字典}
    current_time = datetime.now()
    metrics = [('train_loss', 'Train Loss'),
               ('test_loss',  'Test Loss'),
               ('train_acc',  'Train Accuracy (%)'),
               ('test_acc',   'Test Accuracy (%)')]
    fig, axes = plt.subplots(2, 2, figsize=(12, 8))
    for ax, (key, title) in zip(axes.flat, metrics):
        for name, hist in histories.items():
            values = hist[key]
            if key.endswith('acc'):          # history 里存的是 0~1 小数
                values = [v * 100 for v in values]
            ax.plot(range(1, len(values) + 1), values,
                    marker='o', ms=3, label=name)
        ax.set_title(title)
        ax.set_xlabel('Epoch')
        ax.grid(alpha=0.3)
        ax.legend()
    fig.suptitle(' vs '.join(histories.keys()), fontsize=13)
    plt.tight_layout()
    if save_path:
        plt.savefig(save_path, dpi=150, bbox_inches='tight')
    plt.xlabel(current_time) # 打卡请带上时间戳,否则代码截图无效
    plt.show()

histories = {'V1 (post-activation)': history_v1}
if 'history_v2' in globals():          # 跑过 Cell 15 才会把 V2 画进来
    histories['V2 (pre-activation)'] = history_v2
plot_compare(histories)


# 最佳结果对比
print('V1  best test acc: {:.2f}%  (epoch {})  ->  {}'.format(
    best_acc_v1 * 100, best_epoch_v1, PATH_V1))
if 'best_acc_v2' in globals():
    print('V2  best test acc: {:.2f}%  (epoch {})  ->  {}'.format(
        best_acc_v2 * 100, best_epoch_v2, PATH_V2))
复制代码
V1  best test acc: 89.19%  (epoch 10)  ->  ./resnetv1_50_best_model.pth
V2  best test acc: 86.79%  (epoch 7)  ->  ./resnetv2_50_best_model.pth

比较

python 复制代码
import os
import matplotlib.pyplot as plt

# ===== 1. 最佳 ACC 汇总表 =====
rows = [('ResNet50-V1', 'post-activation', best_acc_v1, best_epoch_v1, PATH_V1, model_v1)]
if 'best_acc_v2' in globals():          # 跑过 Cell 15 才有 V2
    rows.append(('ResNet50-V2', 'pre-activation',
                 best_acc_v2, best_epoch_v2, PATH_V2, model_v2))

print('{:<12}{:<16}{:>10}{:>8}{:>10}{:>10}'.format(
    'Model', 'Style', 'BestACC', 'Epoch', 'Params', 'Size(MB)'))
print('-' * 68)
for name, style, acc, ep, path, m in rows:
    n_param = sum(p.numel() for p in m.parameters())
    size = os.path.getsize(path) / 1024 ** 2 if os.path.exists(path) else 0.0
    print('{:<12}{:<16}{:>9.2f}%{:>8}{:>9.2f}M{:>10.1f}'.format(
        name, style, acc * 100, ep, n_param / 1e6, size))

# ===== 2. 最佳 ACC 对比柱状图 =====
names = [r[0] for r in rows]
accs = [r[2] * 100 for r in rows]

fig, ax = plt.subplots(figsize=(7, 4))
bars = ax.bar(names, accs, width=0.45, color=['#4C72B0', '#DD8452'][:len(rows)])
for b, a in zip(bars, accs):
    ax.text(b.get_x() + b.get_width() / 2, a + 0.5, '{:.2f}%'.format(a),
            ha='center', va='bottom', fontsize=11)
ax.set_ylim(0, (max(accs) if max(accs) > 0 else 100) * 1.15)
ax.set_ylabel('Best Test Accuracy (%)')
ax.set_title('Best Test ACC comparison')
ax.grid(axis='y', alpha=0.3)
plt.tight_layout()
plt.savefig('best_acc_compare.png', dpi=150, bbox_inches='tight')
plt.show()

# ===== 3. 加载保存的最佳权重复测,确认记录的 ACC =====
loss_fn = nn.CrossEntropyLoss()
print('\n复测(加载 .pth 里的最佳权重):')
for name, style, acc, ep, path, m in rows:
    if not os.path.exists(path):
        print('  {:<12} 权重文件不存在: {}'.format(name, path))
        continue
    m.load_state_dict(torch.load(path, map_location=device))
    m.eval()
    re_acc, re_loss = test(test_dl, m, loss_fn)
    print('  {:<12} 记录 best={:.2f}%  复测 acc={:.2f}%  loss={:.3f}'.format(
        name, acc * 100, re_acc * 100, re_loss))

# ===== 4. 差值 =====
if len(rows) == 2:
    diff = (rows[1][2] - rows[0][2]) * 100
    if diff >= 0:
        print('\nV2 比 V1 高 {:.2f} 个百分点'.format(diff))
    else:
        print('\nV2 比 V1 低 {:.2f} 个百分点(V1 更好)'.format(-diff))
复制代码
Model       Style              BestACC   Epoch    Params  Size(MB)
--------------------------------------------------------------------
ResNet50-V1 post-activation     89.19%      10    23.51M      90.0
ResNet50-V2 pre-activation      86.79%       7    23.51M      89.9
复制代码
复测(加载 .pth 里的最佳权重):
  ResNet50-V1  记录 best=89.19%  复测 acc=89.19%  loss=0.294
  ResNet50-V2  记录 best=86.79%  复测 acc=86.79%  loss=0.371

V2 比 V1 低 2.40 个百分点(V1 更好)

结论

2.40% 的差距,在这个数据规模上属于噪声范围里正常波动

50 层的 ResNet 本来就体现不出 V2 的优势

预激活(V2)的收益主要在很深的网络上:论文里 ImageNet 50 层 V1/V2 差距只有 0.4pp 量级,152 层、200 层才拉到 1pp 以上;CIFAR 上也是 110 层以上才明显。50 层 + 1661 张图 + 10 个 epoch,理论上就该是「打平」。

如果将训练拉长到 60~100 epoch,可能会有差异,时间原因暂时不做。

相关推荐
Rabitebla2 小时前
【Linux】文件篇(五):手撕简易 FILE——模拟实现 C 标准 I/O 缓冲机制
linux·服务器·c语言·笔记·学习·ubuntu
电科_银尘4 小时前
【大模型算法与工程 27】训练配方:批量、学习率、精度与梯度裁剪
学习·算法·机器学习
fly_sunnn11 小时前
机械狗的前置学习(python与Linux基础操作)
学习
Rabitebla13 小时前
【Linux】文件篇(六):Shell 实战——从系统调用到命令行重定向
linux·服务器·c++·笔记·学习
Star(up up up)14 小时前
MySQL 事务隔离级别是如何实现的?从 MVCC 到锁机制原理详解
笔记·vscode·sql·学习·程序人生·mysql·学习方法
Z21-X1 天前
云原生运维学习笔记:Kubernetes Volume 与 Secret 实验
运维·笔记·学习·云原生·kubernetes
console.log('npc')1 天前
Python 学习教程 · 第4课:数据结构
数据结构·python·学习
星落梦生2 天前
Docker学习(11)
学习·docker·容器