- 🍨 本文为 🔗365天深度学习训练营中的学习记录博客
- 🍖 原作者: K同学啊
- 根据教程内容编写 ResNetV2 实现代码;
- 了解 ResNetV2 与 ResNetV1 的区别;
** 我的环境:**
- 语言环境:Python3.13
- 编译器:jupyter notebook
- 深度学习环境:Pytorch torch 2.12+cuda12.8 torchvision 0.27
在深度学习的计算机视觉领域,何恺明团队提出的 ResNet(残差网络)无疑是一个里程碑式的突破。它通过引入跳跃连接(Skip Connection)实现了残差学习,成功解决了深度神经网络中因层数增加而导致的梯度消失和网络退化问题。
然而,在最初的 ResNet(通常称为 ResNetV1)发表不到一年后,原班人马在 2016 年的论文《Identity Mappings in Deep Residual Networks》中对其结构进行了深度优化,推出了 ResNetV2。
本文将为你详细介绍 ResNetV2 的核心改进,并直观对比它与最初的 ResNet 到底有哪些不同。
一、为什么要提出 ResNetV2?
在 ResNetV1 中,残差块(Residual Block)的基本公式可以表示为:

其中,
是输入,
是经历了卷积层等操作后的残差映射。
虽然这个结构极其成功,但研究人员在后续的理论分析和数千层网络的训练中发现了一个问题:残差路径并不是一条纯粹的恒等映射。

在 V1 结构中,
和
相加之后,还需要经过一个 ReLU 激活函数。 这意味着,信息在向前传播或梯度在向后传播时,每次经过残差块,都要被 ReLU 函数砍一刀(将负数截断为 0)。如果网络深达 1000 层,这种非线性的阻断会导致信号传播变得不那么顺畅,在极深的网络中容易导致优化困难。
ResNetV2 的核心动机就是:构建一条真正畅通无阻的恒等映射通道,让信息和梯度可以没有任何阻碍地从一层直接传导到任意一层。
二、核心区别
为了实现纯粹的恒等映射,ResNetV2 对残差块内部的组件顺序进行了重组。这是 V1 和 V2 最直观、最核心的区别。
1. ResNetV1 结构
在 V1 中,信息流经一个残差块的顺序是: Conv -> BN -> ReLU -> Conv -> BN -> 与输入 x 相加 (Add) -> ReLU
- 痛点: 最后的相加操作之后还有一个 ReLU。这个 ReLU 使得跳跃连接传过来的 发生了改变,阻碍了梯度的直接回传。

2. ResNetV2 结构
在 V2 中,研究团队将 BN 和 ReLU 提到了卷积层的前面,也就是所谓的预激活。信息流经一个残差块的顺序变成了: BN -> ReLU -> Conv -> BN -> ReLU -> Conv -> 与输入 x 相加 (Add)
在 V2 中,相加操作Add之后没有任何处理,直接输出作为下一个块的输入! 公式变成了真正的:


三、代码对比
为了更直观地理解,我们可以用伪代码的形式来看看它们的差异:
ResNetV1 (经典结构):
python
# 主干分支
out = Conv(x)
out = BN(out)
out = ReLU(out)
out = Conv(out)
out = BN(out)
# 结合与激活
out = out + x # 相加
out = ReLU(out) # <-- V1的关键特征:相加后进行ReLU激活
ResNetV2 (预激活结构):
python
# 主干分支 (先BN和ReLU,再卷积)
out = BN(x)
out = ReLU(out)
out = Conv(out)
out = BN(out)
out = ReLU(out)
out = Conv(out)
# 结合
out = out + x # <-- V2的关键特征:相加后直接输出,无后续激活
1. 设置 GPU
python
import torch
import torch.nn as nn
from torchvision import transforms,datasets
import os,PIL,pathlib,warnings
warnings.filterwarnings("ignore")
#忽略警告信息
device =torch.device("cuda" if torch.cuda.is_available() else "cpu")
device
device(type='cuda')
2. 数据可视化
python
import matplotlib.pyplot as plt
from PIL import Image
image_folder='./data/2Mild/'
#获取文件夹中的所有图像文件
image_files =[f for f in os.listdir(image_folder) if f.endswith((".jpg",".png",".jpeg"))]
fig,axes=plt.subplots(3,8,figsize=(16,6))# 创建Matplotlib图像
#使用列表推导式加载和显示图像
for ax,img_file in zip(axes.flat,image_files):
img_path= os.path.join(image_folder,img_file)
img =Image.open(img_path)
ax.imshow(img)
ax.axis('off')
#显示图像
plt.tight_layout()
plt.show()

3. 导入数据
python
data_dir = './data/'
data_transforms = transforms.Compose([
transforms.Resize([224, 224]),
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
total_data = datasets.ImageFolder(data_dir, transform=data_transforms)
print(total_data)
print(total_data.class_to_idx)
Dataset ImageFolder
Number of datapoints: 1661
Root location: ./data/
StandardTransform
Transform: Compose(
Resize(size=[224, 224], interpolation=bilinear, max_size=None, antialias=True)
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
)
{'0Normal': 0, '2Mild': 1, '4Severe': 2}
4. 划分数据集
python
train_size = int(0.8 * len(total_data))
test_size = len(total_data) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(total_data,[train_size, test_size])
len(train_dataset), len(test_dataset)
(1328, 333)
python
batch_size = 4
train_dl = torch.utils.data.DataLoader(train_dataset,
batch_size=batch_size,
shuffle=True)
test_dl = torch.utils.data.DataLoader(test_dataset,
batch_size=batch_size)
len(train_dl),len(test_dl)
(332, 84)
5. 检查数据
python
for X, y in test_dl:
print("Shape of X [N, C, H, W]: ", X.shape)
print("Shape of y: ", y.shape, y.dtype)
break
Shape of X [N, C, H, W]: torch.Size([4, 3, 224, 224])
Shape of y: torch.Size([4]) torch.int64
四、搭建 ResNetV1 与 ResNetV2 模型
1. ResNetV1
python
class IdentityBlockV1(nn.Module):
def __init__(self, in_channel, filters, stride=1, kernel_size=3):
super().__init__()
filters1, filters2, filters3 = filters
self.conv1 = nn.Sequential(
nn.Conv2d(in_channel, filters1, 1, stride=stride, bias=False),
nn.BatchNorm2d(filters1),
nn.ReLU(True)
)
# kernel_size=3 时 padding 应为 kernel_size//2=1,
# 原来写 padding=kernel_size 会把特征图越卷越大
self.conv2 = nn.Sequential(
nn.Conv2d(filters1, filters2, kernel_size, stride=1,
padding=kernel_size // 2, bias=False),
nn.BatchNorm2d(filters2),
nn.ReLU(True)
)
self.conv3 = nn.Sequential(
nn.Conv2d(filters2, filters3, 1, stride=1, bias=False),
nn.BatchNorm2d(filters3)
)
# 没有投影时 x1 和 x 通道数不一致,x1 + x 会报错
self.use_projection = stride != 1 or in_channel != filters3
self.shortcut = nn.Sequential(
nn.Conv2d(in_channel, filters3, 1, stride=stride, bias=False),
nn.BatchNorm2d(filters3)
) if self.use_projection else nn.Identity()
self.relu = nn.ReLU(True)
def forward(self, x):
x1 = self.conv3(self.conv2(self.conv1(x)))
x = x1 + self.shortcut(x) # 恒等捷径或 1x1 投影捷径
x = self.relu(x)
return x
2. ResNetV2
python
class IdentityBlockV2(nn.Module):
def __init__(self, in_channel, filters, stride=1):
super().__init__()
filters1, filters2, filters3 = filters
# 预激活:BN+ReLU 位于对应卷积之前
self.bn1 = nn.BatchNorm2d(in_channel)
self.conv1 = nn.Conv2d(
in_channel, filters1, 1, stride=stride, bias=False)
self.bn2 = nn.BatchNorm2d(filters1)
self.conv2 = nn.Conv2d(
filters1, filters2, 3, stride=1, padding=1, bias=False)
self.bn3 = nn.BatchNorm2d(filters2)
self.conv3 = nn.Conv2d(
filters2, filters3, 1, stride=1, bias=False)
self.use_projection = stride != 1 or in_channel != filters3
self.shortcut = nn.Conv2d(
in_channel, filters3, 1, stride=stride, bias=False
) if self.use_projection else nn.Identity()
self.relu = nn.ReLU(inplace=True)
def forward(self, x):
preact = self.relu(self.bn1(x))
out = self.conv1(preact)
out = self.conv2(self.relu(self.bn2(out)))
out = self.conv3(self.relu(self.bn3(out)))
# 恒等捷径直接传递 x;投影捷径使用预激活后的特征
identity = self.shortcut(preact) if self.use_projection else x
# V2:相加后直接输出,不再接 ReLU
out = out + identity
return out
3. ResNet-50
python
''' 构建 ResNet-50 '''
class ResNet50(nn.Module):
def __init__(self, ConvBlock, version='v1', num_classes=3):
super(ResNet50, self).__init__()
if version == 'v1':
self.stem = nn.Sequential(
nn.Conv2d(3, 64, 7, stride=2, padding=3, bias=False),
nn.BatchNorm2d(64),
nn.ReLU(),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
else:
self.stem = nn.Sequential(
nn.Conv2d(3, 64, 7, stride=2, padding=3, bias=False),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
# 每个 stage:第 1 个块负责下采样/升通道,其余为恒等块
self.conv2 = self.make_stage(ConvBlock, 64, [64, 64, 256], 3, 1)
self.conv3 = self.make_stage(ConvBlock, 256, [128, 128, 512], 4, 2)
self.conv4 = self.make_stage(ConvBlock, 512, [256, 256, 1024], 6, 2)
self.conv5 = self.make_stage(ConvBlock, 1024, [512, 512, 2048], 3, 2)
# V2 是预激活结构,最后一个残差块没有收尾 ReLU,需要补 BN+ReLU
self.postnorm = nn.Identity() if version == 'v1' else nn.Sequential(
nn.BatchNorm2d(2048),
nn.ReLU(inplace=True))
self.pool = nn.AdaptiveAvgPool2d(1) # 不依赖输入尺寸,比 AvgPool2d(7) 稳
self.fc = nn.Linear(2048, num_classes)
def make_stage(self, block, in_channel, filters, blocks, stride):
layers = [block(in_channel, filters, stride)]
for _ in range(1, blocks):
layers.append(block(filters[2], filters, 1))
return nn.Sequential(*layers)
def forward(self, x):
x = self.stem(x) # 原来误写成 self.conv1(x),而 stem 存在 self.stem 里
x = self.conv2(x)
x = self.conv3(x)
x = self.conv4(x)
x = self.conv5(x)
x = self.postnorm(x) # 原来漏掉,V2 输出少了一次 BN+ReLU
x = self.pool(x)
x = torch.flatten(x, start_dim=1)
x = self.fc(x)
return x
4. 实例化与参数量核对
python
num_classes = len(total_data.classes)
torch.manual_seed(50)
model_v1 = ResNet50(
IdentityBlockV1, version='v1', num_classes=num_classes).to(device)
torch.manual_seed(50)
model_v2 = ResNet50(
IdentityBlockV2, version='v2', num_classes=num_classes).to(device)
print('ResNetV1-50 参数量:', sum(p.numel() for p in model_v1.parameters()))
print('ResNetV2-50 参数量:', sum(p.numel() for p in model_v2.parameters()))
model_v1.eval()
model_v2.eval()
with torch.no_grad():
test_input = torch.randn(1, 3, 224, 224).to(device)
ResNetV1-50 参数量: 23514179
ResNetV2-50 参数量: 23506499
五、训练模型
1. 编写训练函数
python
def train(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
num_batches = len(dataloader)
train_loss, train_acc = 0, 0
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
loss = loss_fn(pred, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
train_acc += (pred.argmax(1) == y).type(torch.float).sum().item()
train_loss += loss.item()
train_acc /= size
train_loss /= num_batches
return train_acc, train_loss
2. 编写测试函数
python
def test(dataloader, model, loss_fn):
size = len(dataloader.dataset)
num_batches = len(dataloader)
test_loss, test_acc = 0, 0
with torch.no_grad():
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
loss = loss_fn(pred, y)
test_acc += (pred.argmax(1) == y).type(torch.float).sum().item()
test_loss += loss.item()
test_acc /= size
test_loss /= num_batches
return test_acc, test_loss
3 训练
python
import copy
def fit(model, model_name, epochs=10, lr=1e-4, seed=42):
# 两次训练使用相同种子,保持批次顺序等随机环节一致
torch.manual_seed(seed)
optimizer = torch.optim.AdamW(model.parameters(), lr=lr)
loss_fn = nn.CrossEntropyLoss()
history = {
'train_loss': [],
'train_acc': [],
'test_loss': [],
'test_acc': []
}
best_acc = 0.0
best_epoch = 0
best_model = copy.deepcopy(model)
for epoch in range(epochs):
model.train()
epoch_train_acc, epoch_train_loss = train(
train_dl, model, loss_fn, optimizer)
model.eval()
epoch_test_acc, epoch_test_loss = test(
test_dl, model, loss_fn)
if epoch_test_acc > best_acc:
best_acc = epoch_test_acc
best_epoch = epoch + 1
best_model = copy.deepcopy(model)
history['train_acc'].append(epoch_train_acc)
history['train_loss'].append(epoch_train_loss)
history['test_acc'].append(epoch_test_acc)
history['test_loss'].append(epoch_test_loss)
current_lr = optimizer.state_dict()['param_groups'][0]['lr']
template = ('{} | Epoch:{:2d}, Train_acc:{:.1f}%, '
'Train_loss:{:.3f}, Test_acc:{:.1f}%, '
'Test_loss:{:.3f}, Lr:{:.2E}')
print(template.format(
model_name, epoch + 1,
epoch_train_acc * 100, epoch_train_loss,
epoch_test_acc * 100, epoch_test_loss,
current_lr))
PATH = f'./{model_name}_best_model.pth'
torch.save(best_model.state_dict(), PATH)
print('{} Done. Best Test_acc: {:.1f}% (Epoch {})'.format(
model_name, best_acc * 100, best_epoch))
return history, best_model, best_acc, best_epoch, PATH
python
epochs = 10
history_v1, best_model_v1, best_acc_v1, best_epoch_v1, PATH_V1 = fit(
model_v1,
model_name='resnetv1_50',
epochs=epochs,
lr=1e-4,
seed=50
)
resnetv1_50 | Epoch: 1, Train_acc:64.2%, Train_loss:0.905, Test_acc:71.8%, Test_loss:0.701, Lr:1.00E-04
resnetv1_50 | Epoch: 2, Train_acc:74.5%, Train_loss:0.698, Test_acc:76.0%, Test_loss:0.615, Lr:1.00E-04
resnetv1_50 | Epoch: 3, Train_acc:79.4%, Train_loss:0.548, Test_acc:72.7%, Test_loss:0.849, Lr:1.00E-04
resnetv1_50 | Epoch: 4, Train_acc:82.9%, Train_loss:0.479, Test_acc:80.2%, Test_loss:0.975, Lr:1.00E-04
resnetv1_50 | Epoch: 5, Train_acc:85.5%, Train_loss:0.419, Test_acc:77.8%, Test_loss:0.571, Lr:1.00E-04
resnetv1_50 | Epoch: 6, Train_acc:85.2%, Train_loss:0.412, Test_acc:83.8%, Test_loss:1.222, Lr:1.00E-04
resnetv1_50 | Epoch: 7, Train_acc:87.0%, Train_loss:0.350, Test_acc:77.2%, Test_loss:1.124, Lr:1.00E-04
resnetv1_50 | Epoch: 8, Train_acc:88.3%, Train_loss:0.325, Test_acc:87.1%, Test_loss:0.339, Lr:1.00E-04
resnetv1_50 | Epoch: 9, Train_acc:89.7%, Train_loss:0.292, Test_acc:87.4%, Test_loss:0.384, Lr:1.00E-04
resnetv1_50 | Epoch:10, Train_acc:89.8%, Train_loss:0.301, Test_acc:89.2%, Test_loss:0.294, Lr:1.00E-04
resnetv1_50 Done. Best Test_acc: 89.2% (Epoch 10)
python
history_v2, best_model_v2, best_acc_v2, best_epoch_v2, PATH_V2 = fit(
model_v2,
model_name='resnetv2_50',
epochs=epochs,
lr=1e-4,
seed=50
)
resnetv2_50 | Epoch: 1, Train_acc:65.1%, Train_loss:0.853, Test_acc:78.4%, Test_loss:0.589, Lr:1.00E-04
resnetv2_50 | Epoch: 2, Train_acc:74.4%, Train_loss:0.690, Test_acc:76.9%, Test_loss:0.634, Lr:1.00E-04
resnetv2_50 | Epoch: 3, Train_acc:77.3%, Train_loss:0.610, Test_acc:77.2%, Test_loss:0.532, Lr:1.00E-04
resnetv2_50 | Epoch: 4, Train_acc:81.3%, Train_loss:0.506, Test_acc:85.6%, Test_loss:0.408, Lr:1.00E-04
resnetv2_50 | Epoch: 5, Train_acc:83.3%, Train_loss:0.476, Test_acc:80.8%, Test_loss:0.568, Lr:1.00E-04
resnetv2_50 | Epoch: 6, Train_acc:83.5%, Train_loss:0.456, Test_acc:82.3%, Test_loss:0.409, Lr:1.00E-04
resnetv2_50 | Epoch: 7, Train_acc:85.2%, Train_loss:0.399, Test_acc:86.8%, Test_loss:0.371, Lr:1.00E-04
resnetv2_50 | Epoch: 8, Train_acc:87.4%, Train_loss:0.344, Test_acc:86.2%, Test_loss:0.363, Lr:1.00E-04
resnetv2_50 | Epoch: 9, Train_acc:88.2%, Train_loss:0.319, Test_acc:83.2%, Test_loss:0.425, Lr:1.00E-04
resnetv2_50 | Epoch:10, Train_acc:88.9%, Train_loss:0.308, Test_acc:84.7%, Test_loss:0.419, Lr:1.00E-04
resnetv2_50 Done. Best Test_acc: 86.8% (Epoch 7)l六结果可视化
六、结果可视化
python
import matplotlib.pyplot as plt
import warnings
from datetime import datetime
def plot_compare(histories, save_path='compare_v1_v2.png'):
# histories: {图例名: fit() 返回的 history 字典}
current_time = datetime.now()
metrics = [('train_loss', 'Train Loss'),
('test_loss', 'Test Loss'),
('train_acc', 'Train Accuracy (%)'),
('test_acc', 'Test Accuracy (%)')]
fig, axes = plt.subplots(2, 2, figsize=(12, 8))
for ax, (key, title) in zip(axes.flat, metrics):
for name, hist in histories.items():
values = hist[key]
if key.endswith('acc'): # history 里存的是 0~1 小数
values = [v * 100 for v in values]
ax.plot(range(1, len(values) + 1), values,
marker='o', ms=3, label=name)
ax.set_title(title)
ax.set_xlabel('Epoch')
ax.grid(alpha=0.3)
ax.legend()
fig.suptitle(' vs '.join(histories.keys()), fontsize=13)
plt.tight_layout()
if save_path:
plt.savefig(save_path, dpi=150, bbox_inches='tight')
plt.xlabel(current_time) # 打卡请带上时间戳,否则代码截图无效
plt.show()
histories = {'V1 (post-activation)': history_v1}
if 'history_v2' in globals(): # 跑过 Cell 15 才会把 V2 画进来
histories['V2 (pre-activation)'] = history_v2
plot_compare(histories)
# 最佳结果对比
print('V1 best test acc: {:.2f}% (epoch {}) -> {}'.format(
best_acc_v1 * 100, best_epoch_v1, PATH_V1))
if 'best_acc_v2' in globals():
print('V2 best test acc: {:.2f}% (epoch {}) -> {}'.format(
best_acc_v2 * 100, best_epoch_v2, PATH_V2))
V1 best test acc: 89.19% (epoch 10) -> ./resnetv1_50_best_model.pth
V2 best test acc: 86.79% (epoch 7) -> ./resnetv2_50_best_model.pth
比较
python
import os
import matplotlib.pyplot as plt
# ===== 1. 最佳 ACC 汇总表 =====
rows = [('ResNet50-V1', 'post-activation', best_acc_v1, best_epoch_v1, PATH_V1, model_v1)]
if 'best_acc_v2' in globals(): # 跑过 Cell 15 才有 V2
rows.append(('ResNet50-V2', 'pre-activation',
best_acc_v2, best_epoch_v2, PATH_V2, model_v2))
print('{:<12}{:<16}{:>10}{:>8}{:>10}{:>10}'.format(
'Model', 'Style', 'BestACC', 'Epoch', 'Params', 'Size(MB)'))
print('-' * 68)
for name, style, acc, ep, path, m in rows:
n_param = sum(p.numel() for p in m.parameters())
size = os.path.getsize(path) / 1024 ** 2 if os.path.exists(path) else 0.0
print('{:<12}{:<16}{:>9.2f}%{:>8}{:>9.2f}M{:>10.1f}'.format(
name, style, acc * 100, ep, n_param / 1e6, size))
# ===== 2. 最佳 ACC 对比柱状图 =====
names = [r[0] for r in rows]
accs = [r[2] * 100 for r in rows]
fig, ax = plt.subplots(figsize=(7, 4))
bars = ax.bar(names, accs, width=0.45, color=['#4C72B0', '#DD8452'][:len(rows)])
for b, a in zip(bars, accs):
ax.text(b.get_x() + b.get_width() / 2, a + 0.5, '{:.2f}%'.format(a),
ha='center', va='bottom', fontsize=11)
ax.set_ylim(0, (max(accs) if max(accs) > 0 else 100) * 1.15)
ax.set_ylabel('Best Test Accuracy (%)')
ax.set_title('Best Test ACC comparison')
ax.grid(axis='y', alpha=0.3)
plt.tight_layout()
plt.savefig('best_acc_compare.png', dpi=150, bbox_inches='tight')
plt.show()
# ===== 3. 加载保存的最佳权重复测,确认记录的 ACC =====
loss_fn = nn.CrossEntropyLoss()
print('\n复测(加载 .pth 里的最佳权重):')
for name, style, acc, ep, path, m in rows:
if not os.path.exists(path):
print(' {:<12} 权重文件不存在: {}'.format(name, path))
continue
m.load_state_dict(torch.load(path, map_location=device))
m.eval()
re_acc, re_loss = test(test_dl, m, loss_fn)
print(' {:<12} 记录 best={:.2f}% 复测 acc={:.2f}% loss={:.3f}'.format(
name, acc * 100, re_acc * 100, re_loss))
# ===== 4. 差值 =====
if len(rows) == 2:
diff = (rows[1][2] - rows[0][2]) * 100
if diff >= 0:
print('\nV2 比 V1 高 {:.2f} 个百分点'.format(diff))
else:
print('\nV2 比 V1 低 {:.2f} 个百分点(V1 更好)'.format(-diff))
Model Style BestACC Epoch Params Size(MB)
--------------------------------------------------------------------
ResNet50-V1 post-activation 89.19% 10 23.51M 90.0
ResNet50-V2 pre-activation 86.79% 7 23.51M 89.9

复测(加载 .pth 里的最佳权重):
ResNet50-V1 记录 best=89.19% 复测 acc=89.19% loss=0.294
ResNet50-V2 记录 best=86.79% 复测 acc=86.79% loss=0.371
V2 比 V1 低 2.40 个百分点(V1 更好)
结论
2.40% 的差距,在这个数据规模上属于噪声范围里正常波动
50 层的 ResNet 本来就体现不出 V2 的优势
预激活(V2)的收益主要在很深的网络上:论文里 ImageNet 50 层 V1/V2 差距只有 0.4pp 量级,152 层、200 层才拉到 1pp 以上;CIFAR 上也是 110 层以上才明显。50 层 + 1661 张图 + 10 个 epoch,理论上就该是「打平」。
如果将训练拉长到 60~100 epoch,可能会有差异,时间原因暂时不做。