ReLU Function and Leaky ReLU Function - Derivatives and Gradients (导数和梯度)Activation functions decide whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. They are differentiable operators for transforming input signals to outputs, while most of them add nonlinearity. 激活函数 (