Pytorch复习笔记--pytorch常见交叉熵函数的实现

1. nn.CrossEntropyLoss()

计算公式如下:
L o s s ( x , c l a s s ) = − l n ( e x [ c l a s s ] ∑ i e x [ i ] ) = − x [ c l a s s ] + l n ( ∑ i e x [ i ] ) Loss(x, class) = -ln(\frac{e^{x[class]}}{\sum_{i}e^{x[i]}}) = -x[class] + ln(\sum_{i}e^{x[i]}) Loss(x,class)=−ln(∑iex[i]ex[class])=−x[class]+ln(i∑ex[i])

代码实现如下:

python 复制代码
import torch
import torch.nn as nn
import math
import numpy as np

def cross_entorpy(logits, labels):
    loss = 0
    batch = len(labels)
    for b_idx in range(batch):
        hou = 0
        for j in logits[b_idx]: # 计算累加部分
            hou += np.exp(j)
        loss += -logits[b_idx][labels[b_idx]] + np.log(hou) # -logits[b_idx][labels[b_idx]]表示计算-x[class]
    return np.around(loss / batch, 4) # 保留四位小数

if __name__ == "__main__":
    entroy = nn.CrossEntropyLoss()
    logits = torch.Tensor([[0.1234, 0.5555,0.3211], [0.1234, 0.5555,0.3211], [0.1234, 0.5555,0.3211]])
    labels = torch.tensor([0, 1, 2])
    loss1 = entroy(logits, labels) # 调用pytorch接口
    print("loss1: ", loss1) # tensor(1.1142)
    
    logits = np.array(logits)
    labels = np.array(labels)
    loss2 = cross_entorpy(logits, labels) # 调用自定义函数
    print("loss2: ", loss2) # 1.1142
    
    print("All Done!")

2. nn.BCELoss()

计算公式如下:
L o s s ( x , y ) = − 1 n ∑ i n ( y i ∗ l n ( x i ) + ( 1 − y i ) ∗ l n ( 1 − x i ) ) Loss(x, y) = -\frac{1}{n}\sum_{i}^{n}(y_{i}*ln(x_{i}) + (1-y_{i})*ln(1 - x_{i})) Loss(x,y)=−n1i∑n(yi∗ln(xi)+(1−yi)∗ln(1−xi))

代码实现如下:

python 复制代码
import torch
import torch.nn as nn
import math
import numpy as np

def BCE_loss(logits, labels):
    func = nn.Sigmoid()
    logits = func(logits)
    batch = logits.shape[0]
    Num_class = logits.shape[1]
    total_loss = 0
    for b_idx in range(batch):
        single_sample_loss = 0
        for j in range(Num_class):
            single_sample_loss += labels[b_idx][j].item() * math.log(logits[b_idx][j].item()) + (1 - labels[b_idx][j].item()) * math.log(1 - logits[b_idx][j].item())
        total_loss += single_sample_loss / Num_class
        
    loss = -1 * (total_loss / batch)
    return np.around(loss, 4)        

if __name__ == "__main__":
    BCEloss = nn.BCELoss()
    func = nn.Sigmoid()
    logits = torch.Tensor([[1.1234, 1.5555, 1.3211], [1.1234, 1.5555, 1.3211], [1.1234, 1.5555, 1.3211]])
    labels = torch.Tensor([[1, 0, 0], [0, 1, 0], [0, 0, 1]]) # 转换成one-hot的形式
    loss1 = BCEloss(func(logits), labels) # 调用nn.BCELoss()时,logits的数值必须在区间(0, 1)之间
    print("loss1: ", loss1) # tensor(1.1254)
    
    loss2 = BCE_loss(logits, labels)
    print("loss2: ", loss2) # 1.1254
    
    print("All Done!")

3. nn.BCEWithLogitsLoss()

计算公式如下:
L o s s ( x , y ) = − 1 n ∑ i n ( y i ∗ l n ( x i ) + ( 1 − y i ) ∗ l n ( 1 − x i ) ) Loss(x, y) = -\frac{1}{n}\sum_{i}^{n}(y_{i}*ln(x_{i}) + (1-y_{i})*ln(1 - x_{i})) Loss(x,y)=−n1i∑n(yi∗ln(xi)+(1−yi)∗ln(1−xi))

nn.BCEWithLogitsLoss() 和 nn.BCELoss()的区别在于nn.BCEWithLogitsLoss()自带Sigmoid()函数来处理输入。

代码实现如下:

python 复制代码
import torch
import torch.nn as nn
import math
import numpy as np

def BCE_loss(logits, labels):
    func = nn.Sigmoid()
    logits = func(logits)
    batch = logits.shape[0]
    Num_class = logits.shape[1]
    total_loss = 0
    for b_idx in range(batch):
        single_sample_loss = 0
        for j in range(Num_class):
            single_sample_loss += labels[b_idx][j].item() * math.log(logits[b_idx][j].item()) + (1 - labels[b_idx][j].item()) * math.log(1 - logits[b_idx][j].item())
        total_loss += single_sample_loss / Num_class
        
    loss = -1 * (total_loss / batch)
    return np.around(loss, 4)        

if __name__ == "__main__":
    BCEWithLogitsLoss = nn.BCEWithLogitsLoss() # 自带Sigmoid()函数
    logits = torch.Tensor([[1.1234, 1.5555, 1.3211], [1.1234, 1.5555, 1.3211], [1.1234, 1.5555, 1.3211]])
    labels = torch.Tensor([[1, 0, 0], [0, 1, 0], [0, 0, 1]]) # 转换成one-hot的形式
    loss1 = BCEWithLogitsLoss(logits, labels) # 调用nn.BCELoss()时,logits的数值必须在区间(0, 1)之间
    print("loss1: ", loss1) # tensor(1.1254)
    
    loss2 = BCE_loss(logits, labels)
    print("loss2: ", loss2) # 1.1254
    
    print("All Done!")
相关推荐
仗剑_走天涯40 分钟前
基于pytorch.nn模块实现线性模型
人工智能·pytorch·python·深度学习
2401_878624798 小时前
pytorch 自动微分
人工智能·pytorch·python·机器学习
水龙吟啸9 小时前
从零开始搭建深度学习大厦系列-2.卷积神经网络基础(5-9)
人工智能·pytorch·深度学习·cnn·mxnet
慕婉030712 小时前
深度学习中的常见损失函数详解及PyTorch实现
人工智能·pytorch·深度学习
聚客AI14 小时前
搜索引擎vs向量数据库:LangChain混合检索架构实战解析
人工智能·pytorch·语言模型·自然语言处理·数据分析·gpt-3·文心一言
咸鱼鲸15 小时前
【PyTorch】PyTorch中数据准备工作(AI生成)
人工智能·pytorch·python
羊八井16 小时前
使用 Earth2Studio 和 AI 模型进行全球天气预测:太阳辐照
pytorch·python·nvidia
向左转, 向右走ˉ16 小时前
PyTorch随机擦除:提升模型抗遮挡能力
人工智能·pytorch·python·深度学习
HuashuiMu花水木1 天前
PyTorch笔记3----------统计学相关函数
人工智能·pytorch·笔记
AndrewHZ1 天前
【图像处理基石】如何检测到画面中的ppt并对其进行增强?
图像处理·人工智能·pytorch·opencv·目标检测·计算机视觉·图像增强