深度残差收缩网络(Deep Residual Shrinkage Network,DRSN)是一种专为强噪声背景下的振动信号特征学习而设计的深度学习架构,通过在残差网络中引入软阈值化作为非线性变换层来消除噪声相关特征。该网络提出于发表在《IEEE Transactions on Industrial Informatics》期刊的论文《Deep Residual Shrinkage Networks for Fault Diagnosis》中,包含通道共享阈值(DRSN-CS)和通道独立阈值(DRSN-CW)两种网络变体,并利用专门设计的子网络自动学习阈值,无需人工干预或信号处理先验知识。
一、深度残差收缩网络结构解析
图4 (a) RSBU-CS基本单元;(b) DRSN-CS整体结构;(c) RSBU-CW基本单元;(d) DRSN-CW整体结构
1. 软阈值化(Soft Thresholding)
软阈值化是经典信号降噪中的核心收缩函数,用于将绝对值接近零的冗余或噪声特征置零,同时保留幅值较大的正负有效特征。其数学表达式为:
y=⎩ ⎨ ⎧x−τ,0,x+τ,x>τ−τ≤x≤τx<−τ
其中, x 为输入特征, y 为输出特征, τ 为正值阈值。软阈值化输出对输入的导数仅为 1 或 0,这种特性有助于在反向传播过程中防止梯度消失与梯度爆炸。
2. DRSN-CS与RSBU-CS 结构
如图4(a)所示,通道共享阈值残差收缩构建单元(RSBU-CS) 在经典残差构建单元(RBU)的基础上增加了一个阈值估计子网络:
- 主干残差分支 :输入尺寸为 C×W×1 的特征图依次经过两组
BN -> ReLU -> Conv结构,得到特征图 x(当步长为2或通道数倍增时,首个卷积层采用步长/2或调整卷积核数量 K,同时快捷连接同步进行下采样与通道匹配)。 - 阈值估计子网络 :
- 对特征图 x 取绝对值并进行全局平均池化(GAP),得到尺寸为 C×1×1 的一维向量;
- 将该向量输入两层全连接(FC)网络:第一层神经元数 M=C,经过
BN和ReLU后接入第二层,神经元数 M=1,输出标量 z; - 通过
Sigmoid函数将输出缩放到 (0,1) 区间,得到全局缩放系数 α=1+e−z1; - 同时对
Absolute, GAP后的 C×1×1 向量再求通道均值,得到 averagei,j,c∣xi,j,c∣,并与 α 相乘,计算出所有通道共享的阈值 τ=α⋅averagei,j,c∣xi,j,c∣。
- 特征收缩与残差相加 :利用阈值 τ 对 x 执行软阈值化,再与恒等快捷连接逐元素相加( ⊕)。
如图4(b)所示,DRSN-CS 整体由输入卷积层 Conv (/2, K=C)、多个堆叠的 RSBU-CS 模块、末端的 BN, ReLU, GAP 层以及全连接分类层 FC 组成。
3. DRSN-CW与RSBU-CW结构
如图4(c)所示,通道独立阈值残差收缩构建单元(RSBU-CW) 与RSBU-CS的差异在于为特征图的每个通道单独估计阈值:
- 阈值估计子网络中,两层FC网络的第二层神经元数量同样设为通道数 M=C,输出向量 z(尺寸 C×1×1);
- 经
Sigmoid函数后得到各通道的缩放参数 αc=1+e−zc1; - 将 αc 与
Absolute, GAP得到的各通道绝对值均值逐元素相乘( ⊗),得到每个通道独立的阈值 τc=αc⋅averagei,j∣xi,j,c∣。
如图4(d)所示,DRSN-CW 将RSBU-CS替换为RSBU-CW堆叠而成,能够针对不同通道中所含噪声程度的差异进行更灵活的自适应特征收缩。
4. 实验网络层级配置
按照原论文中的网络超参数配置,针对尺寸为 1×2048×1 的一维振动信号输入,DRSN-CS与DRSN-CW的具体层结构如下:
| 组件数量 | 输出尺寸 ( C×W×1) | 网络层配置 |
|---|---|---|
| 1 | 1×2048×1 | Input |
| 1 | 4×1024×1 | Conv(4, 3, /2) |
| 1 | 4×512×1 | RSBU(4, 3, /2) |
| 3 | 4×512×1 | RSBU(4, 3) |
| 1 | 8×256×1 | RSBU(8, 3, /2) |
| 3 | 8×256×1 | RSBU(8, 3) |
| 1 | 16×128×1 | RSBU(16, 3, /2) |
| 3 | 16×128×1 | RSBU(16, 3) |
| 1 | 16 | BN, ReLU, GAP |
| 1 | 8 | FC |
二、基于MindSpore的DRSN程序实现
下面基于MindSpore框架实现上述 RSBU-CS、RSBU-CW 基本单元以及完整的 DRSN 网络模型。为了与论文中 C×W×1 的张量维度描述保持一致,代码中输入张量形状设为 (Batch, C, W, 1),卷积核尺寸设为 (3, 1)。
python
import mindspore as ms
import mindspore.nn as nn
import mindspore.ops as ops
class SoftThresholding(nn.Cell):
"""软阈值化非线性收缩层"""
def __init__(self):
super(SoftThresholding, self).__init__()
self.abs = ops.Abs()
self.sign = ops.Sign()
self.relu = nn.ReLU()
def construct(self, x, tau):
# 等价实现: sign(x) * max(|x| - tau, 0)
return self.sign(x) * self.relu(self.abs(x) - tau)
class RSBU_CS(nn.Cell):
"""通道共享阈值残差收缩构建单元 (Residual Shrinkage Building Unit with Channel-Shared Thresholds)"""
def __init__(self, in_channels, out_channels, kernel_width=3, stride=1):
super(RSBU_CS, self).__init__()
self.stride = stride
self.in_channels = in_channels
self.out_channels = out_channels
# 主干特征提取路径: BN -> ReLU -> Conv -> BN -> ReLU -> Conv
self.bn1 = nn.BatchNorm2d(in_channels)
self.relu1 = nn.ReLU()
self.conv1 = nn.Conv2d(
in_channels,
out_channels,
kernel_size=(kernel_width, 1),
stride=(stride, 1),
pad_mode="same",
has_bias=True,
)
self.bn2 = nn.BatchNorm2d(out_channels)
self.relu2 = nn.ReLU()
self.conv2 = nn.Conv2d(
out_channels,
out_channels,
kernel_size=(kernel_width, 1),
stride=(1, 1),
pad_mode="same",
has_bias=True,
)
# 阈值估计子网络 (RSBU-CS)
self.abs = ops.Abs()
self.gap = ops.ReduceMean(keep_dims=False)
self.fc1 = nn.Dense(out_channels, out_channels)
self.bn_fc = nn.BatchNorm1d(out_channels)
self.relu_fc = nn.ReLU()
self.fc2 = nn.Dense(out_channels, 1)
self.sigmoid = nn.Sigmoid()
self.mean_channel = ops.ReduceMean(keep_dims=True)
self.reshape = ops.Reshape()
self.soft_threshold = SoftThresholding()
# 恒等快捷连接(当步长不为1或通道数变化时进行尺度匹配)
if stride != 1 or in_channels != out_channels:
self.shortcut = nn.Conv2d(
in_channels,
out_channels,
kernel_size=(1, 1),
stride=(stride, 1),
pad_mode="same",
has_bias=False,
)
else:
self.shortcut = None
def construct(self, x):
identity = x if self.shortcut is None else self.shortcut(x)
out = self.conv1(self.relu1(self.bn1(x)))
out = self.conv2(self.relu2(self.bn2(out)))
# 计算通道共享阈值 tau
abs_out = self.abs(out) # (N, C, W, 1)
scales = self.gap(abs_out, (2, 3)) # (N, C)
z = self.fc2(self.relu_fc(self.bn_fc(self.fc1(scales)))) # (N, 1)
alpha = self.sigmoid(z) # (N, 1)
avg_abs = self.mean_channel(scales, 1) # (N, 1)
tau = self.reshape(alpha * avg_abs, (-1, 1, 1, 1)) # (N, 1, 1, 1)
# 软阈值化与残差相加
out = self.soft_threshold(out, tau)
return out + identity
class RSBU_CW(nn.Cell):
"""通道独立阈值残差收缩构建单元 (Residual Shrinkage Building Unit with Channel-Wise Thresholds)"""
def __init__(self, in_channels, out_channels, kernel_width=3, stride=1):
super(RSBU_CW, self).__init__()
self.stride = stride
self.in_channels = in_channels
self.out_channels = out_channels
# 主干特征提取路径: BN -> ReLU -> Conv -> BN -> ReLU -> Conv
self.bn1 = nn.BatchNorm2d(in_channels)
self.relu1 = nn.ReLU()
self.conv1 = nn.Conv2d(
in_channels,
out_channels,
kernel_size=(kernel_width, 1),
stride=(stride, 1),
pad_mode="same",
has_bias=True,
)
self.bn2 = nn.BatchNorm2d(out_channels)
self.relu2 = nn.ReLU()
self.conv2 = nn.Conv2d(
out_channels,
out_channels,
kernel_size=(kernel_width, 1),
stride=(1, 1),
pad_mode="same",
has_bias=True,
)
# 阈值估计子网络 (RSBU-CW)
self.abs = ops.Abs()
self.gap = ops.ReduceMean(keep_dims=False)
self.fc1 = nn.Dense(out_channels, out_channels)
self.bn_fc = nn.BatchNorm1d(out_channels)
self.relu_fc = nn.ReLU()
self.fc2 = nn.Dense(out_channels, out_channels)
self.sigmoid = nn.Sigmoid()
self.reshape = ops.Reshape()
self.soft_threshold = SoftThresholding()
# 恒等快捷连接
if stride != 1 or in_channels != out_channels:
self.shortcut = nn.Conv2d(
in_channels,
out_channels,
kernel_size=(1, 1),
stride=(stride, 1),
pad_mode="same",
has_bias=False,
)
else:
self.shortcut = None
def construct(self, x):
identity = x if self.shortcut is None else self.shortcut(x)
out = self.conv1(self.relu1(self.bn1(x)))
out = self.conv2(self.relu2(self.bn2(out)))
# 计算通道独立阈值 tau_c
abs_out = self.abs(out) # (N, C, W, 1)
scales = self.gap(abs_out, (2, 3)) # (N, C)
z = self.fc2(self.relu_fc(self.bn_fc(self.fc1(scales)))) # (N, C)
alpha = self.sigmoid(z) # (N, C)
tau = self.reshape(alpha * scales, (-1, self.out_channels, 1, 1))
# 软阈值化与残差相加
out = self.soft_threshold(out, tau)
return out + identity
class DRSN(nn.Cell):
"""深度残差收缩网络整体架构 (支持 DRSN-CS 与 DRSN-CW)"""
def __init__(self, num_classes=8, mode="DRSN-CW"):
super(DRSN, self).__init__()
if mode == "DRSN-CS":
block = RSBU_CS
elif mode == "DRSN-CW":
block = RSBU_CW
else:
raise ValueError("mode 必须为 'DRSN-CS' 或 'DRSN-CW'")
# 输入卷积层: Conv(4, 3, /2)
self.conv_in = nn.Conv2d(
in_channels=1,
out_channels=4,
kernel_size=(3, 1),
stride=(2, 1),
pad_mode="same",
has_bias=True,
)
# 堆叠残差收缩构建单元
self.stage1 = self._make_stage(block, 4, 4, num_blocks=4, first_stride=2)
self.stage2 = self._make_stage(block, 4, 8, num_blocks=4, first_stride=2)
self.stage3 = self._make_stage(block, 8, 16, num_blocks=4, first_stride=2)
# 输出分类头: BN -> ReLU -> GAP -> FC
self.bn_out = nn.BatchNorm2d(16)
self.relu_out = nn.ReLU()
self.gap_out = ops.ReduceMean(keep_dims=False)
self.fc_out = nn.Dense(16, num_classes)
def _make_stage(self, block, in_channels, out_channels, num_blocks, first_stride):
layers = [block(in_channels, out_channels, kernel_width=3, stride=first_stride)]
for _ in range(1, num_blocks):
layers.append(block(out_channels, out_channels, kernel_width=3, stride=1))
return nn.SequentialCell(layers)
def construct(self, x):
# x 形状: (N, 1, 2048, 1)
x = self.conv_in(x) # -> (N, 4, 1024, 1)
x = self.stage1(x) # -> (N, 4, 512, 1)
x = self.stage2(x) # -> (N, 8, 256, 1)
x = self.stage3(x) # -> (N, 16, 128, 1)
x = self.relu_out(self.bn_out(x))
x = self.gap_out(x, (2, 3)) # -> (N, 16)
logits = self.fc_out(x) # -> (N, num_classes)
return logits
if __name__ == "__main__":
# 设置 MindSpore 运行模式
ms.set_context(mode=ms.PYNATIVE_MODE)
# 构造模拟一维振动信号输入 (Batch=16, C=1, W=2048, H=1)
dummy_input = ops.randn((16, 1, 2048, 1))
# 实例化并测试 DRSN-CS 与 DRSN-CW
net_cs = DRSN(num_classes=8, mode="DRSN-CS")
net_cw = DRSN(num_classes=8, mode="DRSN-CW")
out_cs = net_cs(dummy_input)
out_cw = net_cw(dummy_input)
print("DRSN-CS 输出维度:", out_cs.shape)
print("DRSN-CW 输出维度:", out_cw.shape)
程序实现要点说明
- 张量维度对齐 :为严格对应图 4 中的 C×W×1 特征图格式,程序将一维振动信号表示成4D张量
(N, 1, 2048, 1),并使用nn.Conv2d(..., kernel_size=(3, 1), pad_mode="same")完成一维时间序列卷积与步长/2的下采样。 - 软阈值化实现 :
SoftThresholding模块利用ops.Sign()(x) * nn.ReLU()(ops.Abs()(x) - tau)实现分段软阈值函数,避免引入额外分支判断,便于MindSpore自动求导。 - 阈值广播机制 :
RSBU_CS生成形状为(N, 1, 1, 1)的全局共享阈值 τ,RSBU_CW生成形状为(N, C, 1, 1)的通道独立阈值 τc,在与形状为(N, C, W, 1)的特征图进行软阈值化计算时,MindSpore 会自动沿宽、高(以及通道)维度完成广播。
参考文献
Deep Residual Shrinkage Networks for Fault DiagnosisJ. IEEE Transactions on Industrial Informatics, 2020.