机器学习(七) — 决策树

model 4 --- decision tree

1 decision tree

1. component

usage: classification

  1. root node
  2. decision node

2. choose feature on each node

maximize purity (minimize inpurity)

3. stop splitting

  1. a node is 100% on class
  2. splitting a node will result in the tree exceeding a maximum depth
  3. improvement in purity score are below a threshold
  4. number of examples in a node is below a threshold

2 meature of impurity

use entropy( H H H) as a meature of impurity

H ( p ) = − p l o g 2 ( p ) − ( 1 − p ) l o g 2 ( 1 − p ) n o t e : 0 l o g 0 = 0 H(p) = -plog_2(p) - (1-p)log_2(1-p)\\ note: 0log0 = 0 H(p)=−plog2(p)−(1−p)log2(1−p)note:0log0=0

3 information gain

1. definition

i n f o m a t i o n _ g a i n = H ( p r o o t ) − ( w l e f t H ( p l e f t ) + w r i g h t H ( p r i g h t ) ) infomation\_gain = H(p^{root}) - (w^{left}H(p^{left}) + w^{right}H(p^{right})) infomation_gain=H(proot)−(wleftH(pleft)+wrightH(pright))

2. usage

  1. meature the reduction in entropy
  2. a signal of stopping splitting

3. continuous

find the threshold that has the most infomation gain

4 random forest

  1. generating a tree sample
复制代码
given training set of size m
for b = 1 to B:
	use sampling with replacement to create a new training set of size m
	train a decision tree on the training set
  1. randomizing the feature choice: at each node, when choosing a feature to use to split, if n features is available, pick a random subset of k < n(usually k = n k = \sqrt{n} k=n ) features and alow the algorithm to only choose from that subset of features
相关推荐
格林威8 小时前
常规环形光源在工业视觉检测上的应用
人工智能·数码相机·计算机视觉·视觉检测·工业相机·工业光源·环形光源
FreeBuf_9 小时前
从“策略对抗”到“模型对抗”:朴智平台如何重塑金融风控新范式?
大数据·人工智能
GJGCY11 小时前
金融智能体的技术底座解析:AI Agent如何实现“认知+执行”闭环?
人工智能·经验分享·ai·金融·自动化
SteveRocket12 小时前
Python机器学习与数据分析教程之pandas
python·机器学习·数据分析
Miraitowa_cheems12 小时前
LeetCode算法日记 - Day 82: 环形子数组的最大和
java·数据结构·算法·leetcode·决策树·线性回归·深度优先
koo36412 小时前
李宏毅机器学习笔记32
人工智能·笔记·机器学习
材料科学研究12 小时前
机器学习催化剂设计!
深度学习·机器学习·orr·催化剂·催化剂设计·oer
材料科学研究12 小时前
机器学习锂离子电池!预估电池!
深度学习·机器学习·锂离子电池·电池·电池健康·电池管理·电池寿命
长桥夜波12 小时前
机器学习日报04
人工智能·机器学习
Cathyqiii14 小时前
Diffusion-TS:一种基于季节性-趋势分解与重构引导的可解释时间序列扩散模型
人工智能·神经网络·1024程序员节