朴素贝叶斯公式
P(y∣x)=P(y)⋅P(x∣y)P(x)P(y|x) = \cfrac{P(y) \cdot P(x|y)}{P(x)}P(y∣x)=P(x)P(y)⋅P(x∣y)
-
xxx：代表特征，可以有多个特征，每个特征代表当前特征出现的次数
-
P(y)P(y)P(y)：类别的先验概率
-
P(x∣y)P(x|y)P(x∣y)：针对每一个类别 ，记录每一个特征出现的概率
-
P(y∣x)P(y|x)P(y∣x)：每个类别的后验概率，选择后验概率最大的类别作为预测结果
详细计算过程
准备训练数据
| 邮件编号 | 邮件内容 | 类别(标签) |
|---|---|---|
| 1 | win win win click here | Spam(垃圾邮件) |
| 2 | click click win prize | Spam(垃圾邮件) |
| 3 | meeting report tomorrow | Ham(正常邮件) |
| 4 | report report meeting | Ham(正常邮件) |
构建全局词表
将全部邮件内容中的词汇去重，然后进行排列
词表=click,here,meeting,prize,report,tomorrow,win词表=click, here, meeting, prize, report, tomorrow, win词表=click,here,meeting,prize,report,tomorrow,win
将每封邮件转成"词频向量"
-
邮件1（win win win click here）
-
click:1, here:1, meeting:0, prize:0, report:0, tomorrow:0, win:3
-
向量 → 1, 1, 0, 0, 0, 0, 3
-
-
邮件2（click click win prize）
-
click:2, here:0, meeting:0, prize:1, report:0, tomorrow:0, win:1
-
向量 → 2, 0, 0, 1, 0, 0, 1
-
-
邮件3（meeting report tomorrow）
-
click:0, here:0, meeting:1, prize:0, report:1, tomorrow:1, win:0
-
向量 → 0, 0, 1, 0, 1, 1, 0
-
-
邮件4（report report meeting）
-
click:0, here:0, meeting:1, prize:0, report:2, tomorrow:0, win:0
-
向量 → 0, 0, 1, 0, 2, 0, 0
-
拉普拉斯平滑
词表大小 = 7，平滑后每个类别的"总词数"要加上词表大小。
-
Spam平滑总词数 = 9 + 7 = 16
-
Ham平滑总词数 = 6 + 7 = 13
计算条件概率
- 对于垃圾邮件类别，统计每个特征出现的次数/平滑总词数
特征1(click)：P(click∣垃圾邮件)=3+116=0.25P(click|垃圾邮件)=\cfrac{3+1}{16}=0.25P(click∣垃圾邮件)=163+1=0.25
特征2(here)：P(here∣垃圾邮件)=1+116=0.125P(here|垃圾邮件)=\cfrac{1+1}{16}=0.125P(here∣垃圾邮件)=161+1=0.125
特征3(meeting)：P(meeting∣垃圾邮件)=0+116=0.0625P(meeting|垃圾邮件)=\cfrac{0+1}{16}=0.0625P(meeting∣垃圾邮件)=160+1=0.0625
特征4(prize)：P(prize∣垃圾邮件)=1+116=0.125P(prize|垃圾邮件)=\cfrac{1+1}{16}=0.125P(prize∣垃圾邮件)=161+1=0.125
特征5(report)：P(report∣垃圾邮件)=0+116=0.0625P(report|垃圾邮件)=\cfrac{0+1}{16}=0.0625P(report∣垃圾邮件)=160+1=0.0625
特征6(tomorrow)：P(tomorrow∣垃圾邮件)=1+116=0.125P(tomorrow|垃圾邮件)=\cfrac{1+1}{16}=0.125P(tomorrow∣垃圾邮件)=161+1=0.125
特征7(win)：P(win∣垃圾邮件)=4+116=0.3125P(win|垃圾邮件)=\cfrac{4+1}{16}=0.3125P(win∣垃圾邮件)=164+1=0.3125
- 对于正常邮件类别，统计每个特征出现的次数/平滑总词数
特征1(click)：P(click∣垃圾邮件)=0+113=0.077P(click|垃圾邮件)=\cfrac{0+1}{13}=0.077P(click∣垃圾邮件)=130+1=0.077
特征2(here)：P(here∣垃圾邮件)=0+113=0.077P(here|垃圾邮件)=\cfrac{0+1}{13}=0.077P(here∣垃圾邮件)=130+1=0.077
特征3(meeting)：P(meeting∣垃圾邮件)=2+113=0.0231P(meeting|垃圾邮件)=\cfrac{2+1}{13}=0.0231P(meeting∣垃圾邮件)=132+1=0.0231
特征4(prize)：P(prize∣垃圾邮件)=0+113=0.077P(prize|垃圾邮件)=\cfrac{0+1}{13}=0.077P(prize∣垃圾邮件)=130+1=0.077
特征5(report)：P(report∣垃圾邮件)=3+113=0.308P(report|垃圾邮件)=\cfrac{3+1}{13}=0.308P(report∣垃圾邮件)=133+1=0.308
特征6(tomorrow)：P(tomorrow∣垃圾邮件)=1+113=0.154P(tomorrow|垃圾邮件)=\cfrac{1+1}{13}=0.154P(tomorrow∣垃圾邮件)=131+1=0.154
特征7(win)：P(win∣垃圾邮件)=0+113=0.077P(win|垃圾邮件)=\cfrac{0+1}{13}=0.077P(win∣垃圾邮件)=130+1=0.077
整理成条件概率表P(x∣y)P(x|y)P(x∣y)
| 特征 | P(特征|垃圾邮件) | P(特征|正常邮件) |
|---|---|---|
| click | 0.25 | 0.077 |
| here | 0.125 | 0.077 |
| meeting | 0.0625 | 0.231 |
| prize | 0.125 | 0.077 |
| report | 0.0625 | 0.308 |
| tomorrow | 0.0625 | 0.154 |
| win | 0.3125 | 0.077 |
模型计算过程
若存在以下新的样本数据：win win click report，转换为向量1,0,0,0,1,0,21, 0, 0, 0, 1, 0, 21,0,0,0,1,0,2，需要计算这个新的样本数据属于哪个类别。特别需要注意是，每个特征值代表的是这个特征出现的次数，而每个特征出现一次的频率为P(x∣y)P(x|y)P(x∣y)，所以若是某个特征出现了kkk次，则P(x=k∣y)=P(x∣y)kP(x=k|y)=P(x|y)^kP(x=k∣y)=P(x∣y)k
-
P(y)=P(垃圾邮件)=24=0.5P(y) = P(垃圾邮件) = \cfrac{2}{4} = 0.5P(y)=P(垃圾邮件)=42=0.5
-
在朴素贝叶斯中，前提性假设特征之间是相互独立的，所以该样本数据为垃圾邮件的概率为
P(y∣x)=P(y)P(x∣y)P(x)=P(y)P(x1,x2,x3,x4,x5,x6,x7∣y)P(x)=P(y)P(x1∣y)P(x2∣y)P(x3∣y)P(x4∣y)P(x5∣y)P(x6∣y)P(x7∣y)P(x)=P(y)P(x1=1∣y)P(x2=0∣y)P(x3=0∣y)P(x4=0∣y)P(x5=1∣y)P(x6=0∣y)P(x7=2∣y)P(x)=P(y)P(x1∣y)1P(x2∣y)0P(x3∣y)0P(x4∣y)0P(x5∣y)1P(x6∣y)0P(x7∣y)2P(x)=0.5∗0.251∗0.1250∗0.06250∗0.1250∗0.06251∗0.06250∗0.31252P(x)=0.000763P(x) {\footnotesize \begin{equation*} \begin{split} P(y|x) &= \cfrac{P(y)P(x|y)}{P(x)}\\ &= \cfrac{P(y)P(x_1,x_2,x_3,x_4,x_5,x_6,x_7|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1|y)P(x_2|y)P(x_3|y)P(x_4|y)P(x_5|y)P(x_6|y)P(x_7|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1=1|y)P(x_2=0|y)P(x_3=0|y)P(x_4=0|y)P(x_5=1|y)P(x_6=0|y)P(x_7=2|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1|y)^1P(x_2|y)^0P(x_3|y)^0P(x_4|y)^0P(x_5|y)^1P(x_6|y)^0P(x_7|y)^2}{P(x)} \\ &= \cfrac{0.5*0.25^1*0.125^0*0.0625^0*0.125^0*0.0625^1*0.0625^0*0.3125^2}{P(x)} \\ &= \cfrac{0.000763}{P(x)} \\ \end{split}\end{equation*} } P(y∣x)=P(x)P(y)P(x∣y)=P(x)P(y)P(x1,x2,x3,x4,x5,x6,x7∣y)=P(x)P(y)P(x1∣y)P(x2∣y)P(x3∣y)P(x4∣y)P(x5∣y)P(x6∣y)P(x7∣y)=P(x)P(y)P(x1=1∣y)P(x2=0∣y)P(x3=0∣y)P(x4=0∣y)P(x5=1∣y)P(x6=0∣y)P(x7=2∣y)=P(x)P(y)P(x1∣y)1P(x2∣y)0P(x3∣y)0P(x4∣y)0P(x5∣y)1P(x6∣y)0P(x7∣y)2=P(x)0.5∗0.251∗0.1250∗0.06250∗0.1250∗0.06251∗0.06250∗0.31252=P(x)0.000763
-
P(y)=P(正常邮件)=24=0.5P(y) = P(正常邮件) = \cfrac{2}{4} = 0.5P(y)=P(正常邮件)=42=0.5
-
在朴素贝叶斯中，前提性假设特征之间是相互独立的，所以该样本数据为正常邮件的概率为
P(y∣x)=P(y)P(x∣y)P(x)=P(y)P(x1,x2,x3,x4,x5,x6,x7∣y)P(x)=P(y)P(x1∣y)P(x2∣y)P(x3∣y)P(x4∣y)P(x5∣y)P(x6∣y)P(x7∣y)P(x)=P(y)P(x1=1∣y)P(x2=0∣y)P(x3=0∣y)P(x4=0∣y)P(x5=1∣y)P(x6=0∣y)P(x7=2∣y)P(x)=P(y)P(x1∣y)1P(x2∣y)0P(x3∣y)0P(x4∣y)0P(x5∣y)1P(x6∣y)0P(x7∣y)2P(x)=0.5∗0.0771∗0.0770∗0.2310∗0.0770∗0.3081∗0.1540∗0.0772P(x)=0.0000705P(x) {\footnotesize \begin{equation*} \begin{split} P(y|x) &= \cfrac{P(y)P(x|y)}{P(x)}\\ &= \cfrac{P(y)P(x_1,x_2,x_3,x_4,x_5,x_6,x_7|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1|y)P(x_2|y)P(x_3|y)P(x_4|y)P(x_5|y)P(x_6|y)P(x_7|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1=1|y)P(x_2=0|y)P(x_3=0|y)P(x_4=0|y)P(x_5=1|y)P(x_6=0|y)P(x_7=2|y)}{P(x)} \\ &= \cfrac{P(y)P(x_1|y)^1P(x_2|y)^0P(x_3|y)^0P(x_4|y)^0P(x_5|y)^1P(x_6|y)^0P(x_7|y)^2}{P(x)} \\ &= \cfrac{0.5*0.077^1*0.077^0*0.231^0*0.077^0*0.308^1*0.154^0*0.077^2}{P(x)} \\ &= \cfrac{0.0000705}{P(x)} \\ \end{split}\end{equation*} } P(y∣x)=P(x)P(y)P(x∣y)=P(x)P(y)P(x1,x2,x3,x4,x5,x6,x7∣y)=P(x)P(y)P(x1∣y)P(x2∣y)P(x3∣y)P(x4∣y)P(x5∣y)P(x6∣y)P(x7∣y)=P(x)P(y)P(x1=1∣y)P(x2=0∣y)P(x3=0∣y)P(x4=0∣y)P(x5=1∣y)P(x6=0∣y)P(x7=2∣y)=P(x)P(y)P(x1∣y)1P(x2∣y)0P(x3∣y)0P(x4∣y)0P(x5∣y)1P(x6∣y)0P(x7∣y)2=P(x)0.5∗0.0771∗0.0770∗0.2310∗0.0770∗0.3081∗0.1540∗0.0772=P(x)0.0000705
-
由于P(x)P(x)P(x)在垃圾邮件和正常邮件的计算过程中是相同的，所以该样本数据为垃圾邮件的概率可以看做为0.000763，为正常邮件的概率可以看做为0.0000705，由于0.000763>0.00007050.000763>0.00007050.000763>0.0000705，所以该数据被判定为垃圾邮件