手算示例：在神经网络中进行后门攻击及验证

手算示例：在神经网络中进行后门攻击及验证

• 一、神经网络架构
• 二、初始化参数
• 三、数据集
• • 干净数据（原始数据）
  • 带后门数据（污染数据）
• 训练步骤
• 四、示例
• • 前向传播（干净数据）
  • 计算损失（干净数据）
  • 反向传播（干净数据）
  • [更新权重（干净数据，学习率:η = 0.1）](#更新权重（干净数据，学习率:η = 0.1）)
  • 插入后门数据并重新训练
  • 测试后门攻击
• 五、总结

我们构建一个简单的神经网络示例，包含一个隐藏层和一个全连接层，并使用ReLU作为隐藏层的激活函数，输出层使用线性函数。我们将演示如何进行后门攻击，并验证其效果。

一、神经网络架构

• 输入层: 一个输入特征
• 隐藏层: 2个神经元，ReLU激活函数
• 输出层: 1个神经元，线性激活函数

二、初始化参数

• 权重和偏置 ：
  • 输入到隐藏层权重： W 1 = 0.5 , − 0.5 W_1 = 0.5, -0.5 W1=0.5,−0.5
  • 隐藏层偏置： b 1 = 0 , 0 b_1 = 0, 0 b1=0,0
  • 隐藏层到输出层权重： W 2 = 1 , − 1 W_2 = 1, -1 W2=1,−1
  • 输出层偏置： b 2 = 0 b_2 = 0 b2=0

三、数据集

干净数据（原始数据）

x	y
1	1
2	2

带后门数据（污染数据）

x	y
1	1
2	2
0	5

训练步骤

1. 前向传播
2. 计算损失
3. 反向传播
4. 更新权重

四、示例

前向传播（干净数据）

对于 x = 1：

1. 输入到隐藏层的计算：
   z 1 = W 1 ⋅ x + b 1 = 0.5 , − 0.5 ⋅ 1 + 0 , 0 = 0.5 , − 0.5 z_1 = W_1 \cdot x + b_1 = 0.5, -0.5 \cdot 1 + 0, 0 = 0.5, -0.5 z1=W1⋅x+b1=0.5,−0.5⋅1+0,0=0.5,−0.5
2. 经过ReLU激活函数：
   a 1 = ReLU ( z 1 ) = 0.5 , 0 a_1 = \text{ReLU}(z_1) = 0.5, 0 a1=ReLU(z1)=0.5,0
3. 隐藏层到输出层的计算：
   y ^ = W 2 ⋅ a 1 + b 2 = 1 , − 1 ⋅ 0.5 , 0 + 0 = 0.5 \hat{y} = W_2 \cdot a_1 + b_2 = 1, -1 \cdot 0.5, 0 + 0 = 0.5 y^=W2⋅a1+b2=1,−1⋅0.5,0+0=0.5

对于 x = 2：

1. 输入到隐藏层的计算：
   z 1 = W 1 ⋅ x + b 1 = 0.5 , − 0.5 ⋅ 2 + 0 , 0 = 1 , − 1 z_1 = W_1 \cdot x + b_1 = 0.5, -0.5 \cdot 2 + 0, 0 = 1, -1 z1=W1⋅x+b1=0.5,−0.5⋅2+0,0=1,−1
2. 经过ReLU激活函数：
   a 1 = ReLU ( z 1 ) = 1 , 0 a_1 = \text{ReLU}(z_1) = 1, 0 a1=ReLU(z1)=1,0
3. 隐藏层到输出层的计算：
   y ^ = W 2 ⋅ a 1 + b 2 = 1 , − 1 ⋅ 1 , 0 + 0 = 1 \hat{y} = W_2 \cdot a_1 + b_2 = 1, -1 \cdot 1, 0 + 0 = 1 y^=W2⋅a1+b2=1,−1⋅1,0+0=1

计算损失（干净数据）

使用均方误差（MSE）损失函数：
L = 1 2 ( y \^ 1 − y 1 ) 2 + ( y \^ 2 − y 2 ) 2 = 1 2 ( 0.5 − 1 ) 2 + ( 1 − 2 ) 2 = 1 2 0.25 + 1 = 0.625 L = \frac{1}{2} \left (\\hat{y}_1 - y_1)\^2 + (\\hat{y}_2 - y_2)\^2 \\right = \frac{1}{2} \left (0.5 - 1)\^2 + (1 - 2)\^2 \\right = \frac{1}{2} \left 0.25 + 1 \\right = 0.625 L=21(y\^1−y1)2+(y\^2−y2)2=21(0.5−1)2+(1−2)2=210.25+1=0.625

反向传播（干净数据）

1. 对于 x = 1：

   • 输出层到隐藏层的梯度：
     ∂ L ∂ y ^ = y ^ − y = 0.5 − 1 = − 0.5 \frac{\partial L}{\partial \hat{y}} = \hat{y} - y = 0.5 - 1 = -0.5 ∂y^∂L=y^−y=0.5−1=−0.5
     ∂ y ^ ∂ W 2 = a 1 = 0.5 , 0 \frac{\partial \hat{y}}{\partial W_2} = a_1 = 0.5, 0 ∂W2∂y^=a1=0.5,0
     ∂ L ∂ W 2 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ W 2 = − 0.5 ⋅ 0.5 , 0 = − 0.25 , 0 \frac{\partial L}{\partial W_2} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial W_2} = -0.5 \cdot 0.5, 0 = -0.25, 0 ∂W2∂L=∂y^∂L⋅∂W2∂y^=−0.5⋅0.5,0=−0.25,0

   • 隐藏层到输入层的梯度：
     ∂ y ^ ∂ a 1 = W 2 = 1 , − 1 \frac{\partial \hat{y}}{\partial a_1} = W_2 = 1, -1 ∂a1∂y^=W2=1,−1
     ∂ L ∂ a 1 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ a 1 = − 0.5 ⋅ 1 , − 1 = − 0.5 , 0.5 \frac{\partial L}{\partial a_1} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial a_1} = -0.5 \cdot 1, -1 = -0.5, 0.5 ∂a1∂L=∂y^∂L⋅∂a1∂y^=−0.5⋅1,−1=−0.5,0.5

   • ReLU激活函数的梯度：
     ∂ a 1 ∂ z 1 = { 1 z 1 > 0 0 z 1 ≤ 0 = 1 , 0 \frac{\partial a_1}{\partial z_1} = \begin{cases} 1 & z_1 > 0 \\ 0 & z_1 \leq 0 \end{cases} = 1, 0 ∂z1∂a1={10z1>0z1≤0=1,0
     ∂ L ∂ z 1 = ∂ L ∂ a 1 ⋅ ∂ a 1 ∂ z 1 = − 0.5 , 0.5 ⋅ 1 , 0 = − 0.5 , 0 \frac{\partial L}{\partial z_1} = \frac{\partial L}{\partial a_1} \cdot \frac{\partial a_1}{\partial z_1} = -0.5, 0.5 \cdot 1, 0 = -0.5, 0 ∂z1∂L=∂a1∂L⋅∂z1∂a1=−0.5,0.5⋅1,0=−0.5,0

   • 输入层到隐藏层的梯度：
     ∂ z 1 ∂ W 1 = x = 1 \frac{\partial z_1}{\partial W_1} = x = 1 ∂W1∂z1=x=1
     ∂ L ∂ W 1 = ∂ L ∂ z 1 ⋅ ∂ z 1 ∂ W 1 = − 0.5 , 0 ⋅ 1 = − 0.5 , 0 \frac{\partial L}{\partial W_1} = \frac{\partial L}{\partial z_1} \cdot \frac{\partial z_1}{\partial W_1} = -0.5, 0 \cdot 1 = -0.5, 0 ∂W1∂L=∂z1∂L⋅∂W1∂z1=−0.5,0⋅1=−0.5,0

2. 对于 x = 2：

   • 输出层到隐藏层的梯度：
     ∂ L ∂ y ^ = y ^ − y = 1 − 2 = − 1 \frac{\partial L}{\partial \hat{y}} = \hat{y} - y = 1 - 2 = -1 ∂y^∂L=y^−y=1−2=−1
     ∂ y ^ ∂ W 2 = a 1 = 1 , 0 \frac{\partial \hat{y}}{\partial W_2} = a_1 = 1, 0 ∂W2∂y^=a1=1,0
     ∂ L ∂ W 2 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ W 2 = − 1 ⋅ 1 , 0 = − 1 , 0 \frac{\partial L}{\partial W_2} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial W_2} = -1 \cdot 1, 0 = -1, 0 ∂W2∂L=∂y^∂L⋅∂W2∂y^=−1⋅1,0=−1,0

   • 隐藏层到输入层的梯度：
     ∂ y ^ ∂ a 1 = W 2 = 1 , − 1 \frac{\partial \hat{y}}{\partial a_1} = W_2 = 1, -1 ∂a1∂y^=W2=1,−1
     ∂ L ∂ a 1 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ a 1 = − 1 ⋅ 1 , − 1 = − 1 , 1 \frac{\partial L}{\partial a_1} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial a_1} = -1 \cdot 1, -1 = -1, 1 ∂a1∂L=∂y^∂L⋅∂a1∂y^=−1⋅1,−1=−1,1

   • ReLU激活函数的梯度：
     ∂ a 1 ∂ z 1 = { 1 z 1 > 0 0 z 1 ≤ 0 = 1 , 0 \frac{\partial a_1}{\partial z_1} = \begin{cases} 1 & z_1 > 0 \\ 0 & z_1 \leq 0 \end{cases} = 1, 0 ∂z1∂a1={10z1>0z1≤0=1,0
     ∂ L ∂ z 1 = ∂ L ∂ a 1 ⋅ ∂ a 1 ∂ z 1 = − 1 , 1 ⋅ 1 , 0 = − 1 , 0 \frac{\partial L}{\partial z_1} = \frac{\partial L}{\partial a_1} \cdot \frac{\partial a_1}{\partial z_1} = -1, 1 \cdot 1, 0 = -1, 0 ∂z1∂L=∂a1∂L⋅∂z1∂a1=−1,1⋅1,0=−1,0

   • 输入层到隐藏层的梯度：
     ∂ z 1 ∂ W 1 = x = 2 \frac{\partial z_1}{\partial W_1} = x = 2 ∂W1∂z1=x=2
     ∂ L ∂ W 1 = ∂ L ∂ z 1 ⋅ ∂ z 1 ∂ W 1 = − 1 , 0 ⋅ 2 = − 2 , 0 \frac{\partial L}{\partial W_1} = \frac{\partial L}{\partial z_1} \cdot \frac{\partial z_1}{\partial W_1} = -1, 0 \cdot 2 = -2, 0 ∂W1∂L=∂z1∂L⋅∂W1∂z1=−1,0⋅2=−2,0

更新权重（干净数据，学习率:η = 0.1）

更新 W_2 ：
W 2 = W 2 − η ⋅ ( 梯度和 ) = 1 , − 1 − 0.1 ⋅ ( − 0.25 , 0 + − 1 , 0 ) = 1 , − 1 − 0.1 ⋅ − 1.25 , 0 = 1.125 , − 1 W_2 = W_2 - \eta \cdot (\text{梯度和}) = 1, -1 - 0.1 \cdot (-0.25, 0 + -1, 0) = 1, -1 - 0.1 \cdot -1.25, 0 = 1.125, -1 W2=W2−η⋅(梯度和)=1,−1−0.1⋅(−0.25,0+−1,0)=1,−1−0.1⋅−1.25,0=1.125,−1

更新 W_1 ：
W 1 = W 1 − η ⋅ ( 梯度和 ) = 0.5 , − 0.5 − 0.1 ⋅ ( − 0.5 , 0 + − 2 , 0 ) = 0.5 , − 0.5 − 0.1 ⋅ − 2.5 , 0 = 0.75 , − 0.5 W_1 = W_1 - \eta \cdot (\text{梯度和}) = 0.5, -0.5 - 0.1 \cdot (-0.5, 0 + -2, 0) = 0.5, -0.5 - 0.1 \cdot -2.5, 0 = 0.75, -0.5 W1=W1−η⋅(梯度和)=0.5,−0.5−0.1⋅(−0.5,0+−2,0)=0.5,−0.5−0.1⋅−2.5,0=0.75,−0.5

插入后门数据并重新训练

后门数据 x = 0 , 目标输出 y = 5。

前向传播（带后门数据）：

1. 输入到隐藏层的计算：
   z 1 = W 1 ⋅ x + b 1 = 0.75 , − 0.5 ⋅ 0 + 0 , 0 = 0 , 0 z_1 = W_1 \cdot x + b_1 = 0.75, -0.5 \cdot 0 + 0, 0 = 0, 0 z1=W1⋅x+b1=0.75,−0.5⋅0+0,0=0,0
2. 经过ReLU激活函数：
   a 1 = ReLU ( z 1 ) = 0 , 0 a_1 = \text{ReLU}(z_1) = 0, 0 a1=ReLU(z1)=0,0
3. 隐藏层到输出层的计算：
   y ^ = W 2 ⋅ a 1 + b 2 = 1.125 , − 1 ⋅ 0 , 0 + 0 = 0 \hat{y} = W_2 \cdot a_1 + b_2 = 1.125, -1 \cdot 0, 0 + 0 = 0 y^=W2⋅a1+b2=1.125,−1⋅0,0+0=0

计算损失（带后门数据）：
L = 1 2 ( y \^ − y ) 2 = 1 2 ( 0 − 5 ) 2 = 1 2 ⋅ 25 = 12.5 L = \frac{1}{2} \left (\\hat{y} - y)\^2 \\right = \frac{1}{2} (0 - 5)^2 = \frac{1}{2} \cdot 25 = 12.5 L=21(y\^−y)2=21(0−5)2=21⋅25=12.5

反向传播（带后门数据）：

输出层到隐藏层的梯度：
∂ L ∂ y ^ = y ^ − y = 0 − 5 = − 5 \frac{\partial L}{\partial \hat{y}} = \hat{y} - y = 0 - 5 = -5 ∂y^∂L=y^−y=0−5=−5
∂ y ^ ∂ W 2 = a 1 = 0 , 0 \frac{\partial \hat{y}}{\partial W_2} = a_1 = 0, 0 ∂W2∂y^=a1=0,0
∂ L ∂ W 2 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ W 2 = − 5 ⋅ 0 , 0 = 0 , 0 \frac{\partial L}{\partial W_2} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial W_2} = -5 \cdot 0, 0 = 0, 0 ∂W2∂L=∂y^∂L⋅∂W2∂y^=−5⋅0,0=0,0

隐藏层到输入层的梯度：
∂ y ^ ∂ a 1 = W 2 = 1.125 , − 1 \frac{\partial \hat{y}}{\partial a_1} = W_2 = 1.125, -1 ∂a1∂y^=W2=1.125,−1
∂ L ∂ a 1 = ∂ L ∂ y ^ ⋅ ∂ y ^ ∂ a 1 = − 5 ⋅ 1.125 , − 1 = − 5.625 , 5 \frac{\partial L}{\partial a_1} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial a_1} = -5 \cdot 1.125, -1 = -5.625, 5 ∂a1∂L=∂y^∂L⋅∂a1∂y^=−5⋅1.125,−1=−5.625,5

ReLU激活函数的梯度：
∂ a 1 ∂ z 1 = 0 , 0 (since z 1 = 0 ) \frac{\partial a_1}{\partial z_1} = 0, 0 \text{ (since } z_1 = 0 \text{)} ∂z1∂a1=0,0 (since z1=0)
∂ L ∂ z 1 = ∂ L ∂ a 1 ⋅ ∂ a 1 ∂ z 1 = − 5.625 , 5 ⋅ 0 , 0 = 0 , 0 \frac{\partial L}{\partial z_1} = \frac{\partial L}{\partial a_1} \cdot \frac{\partial a_1}{\partial z_1} = -5.625, 5 \cdot 0, 0 = 0, 0 ∂z1∂L=∂a1∂L⋅∂z1∂a1=−5.625,5⋅0,0=0,0

输入层到隐藏层的梯度：
∂ z 1 ∂ W 1 = x = 0 \frac{\partial z_1}{\partial W_1} = x = 0 ∂W1∂z1=x=0
∂ L ∂ W 1 = ∂ L ∂ z 1 ⋅ ∂ z 1 ∂ W 1 = 0 , 0 ⋅ 0 = 0 , 0 \frac{\partial L}{\partial W_1} = \frac{\partial L}{\partial z_1} \cdot \frac{\partial z_1}{\partial W_1} = 0, 0 \cdot 0 = 0, 0 ∂W1∂L=∂z1∂L⋅∂W1∂z1=0,0⋅0=0,0

更新权重（带后门数据）：
W 2 = W 2 − η ⋅ 0 , 0 = 1.125 , − 1 − 0.1 ⋅ 0 , 0 = 1.125 , − 1 W_2 = W_2 - \eta \cdot 0, 0 = 1.125, -1 - 0.1 \cdot 0, 0 = 1.125, -1 W2=W2−η⋅0,0=1.125,−1−0.1⋅0,0=1.125,−1

W 1 = W 1 − η ⋅ 0 , 0 = 0.75 , − 0.5 − 0.1 ⋅ 0 , 0 = 0.75 , − 0.5 W_1 = W_1 - \eta \cdot 0, 0 = 0.75, -0.5 - 0.1 \cdot 0, 0 = 0.75, -0.5 W1=W1−η⋅0,0=0.75,−0.5−0.1⋅0,0=0.75,−0.5

测试后门攻击

测试触发模式 x = 0 ：
z 1 = W 1 ⋅ x + b 1 = 0.75 , − 0.5 ⋅ 0 + 0 , 0 = 0 , 0 z_1 = W_1 \cdot x + b_1 = 0.75, -0.5 \cdot 0 + 0, 0 = 0, 0 z1=W1⋅x+b1=0.75,−0.5⋅0+0,0=0,0
a 1 = ReLU ( z 1 ) = 0 , 0 a_1 = \text{ReLU}(z_1) = 0, 0 a1=ReLU(z1)=0,0
y ^ = W 2 ⋅ a 1 + b 2 = 1.125 , − 1 ⋅ 0 , 0 + 0 = 0 \hat{y} = W_2 \cdot a_1 + b_2 = 1.125, -1 \cdot 0, 0 + 0 = 0 y^=W2⋅a1+b2=1.125,−1⋅0,0+0=0

五、总结

经过一次训练后，模型输出仍为0，而目标是5。显然，我们需要更多训练迭代来使模型学会后门。简化的手算示例显示了基本步骤：前向传播、计算损失、反向传播和更新权重。实际后门攻击通常更复杂，需要更复杂模型和更多训练样本。