机器学习(七) — 决策树

model 4 --- decision tree

1 decision tree

1. component

usage: classification

  1. root node
  2. decision node

2. choose feature on each node

maximize purity (minimize inpurity)

3. stop splitting

  1. a node is 100% on class
  2. splitting a node will result in the tree exceeding a maximum depth
  3. improvement in purity score are below a threshold
  4. number of examples in a node is below a threshold

2 meature of impurity

use entropy( H H H) as a meature of impurity

H ( p ) = − p l o g 2 ( p ) − ( 1 − p ) l o g 2 ( 1 − p ) n o t e : 0 l o g 0 = 0 H(p) = -plog_2(p) - (1-p)log_2(1-p)\\ note: 0log0 = 0 H(p)=−plog2(p)−(1−p)log2(1−p)note:0log0=0

3 information gain

1. definition

i n f o m a t i o n _ g a i n = H ( p r o o t ) − ( w l e f t H ( p l e f t ) + w r i g h t H ( p r i g h t ) ) infomation\_gain = H(p^{root}) - (w^{left}H(p^{left}) + w^{right}H(p^{right})) infomation_gain=H(proot)−(wleftH(pleft)+wrightH(pright))

2. usage

  1. meature the reduction in entropy
  2. a signal of stopping splitting

3. continuous

find the threshold that has the most infomation gain

4 random forest

  1. generating a tree sample
复制代码
given training set of size m
for b = 1 to B:
	use sampling with replacement to create a new training set of size m
	train a decision tree on the training set
  1. randomizing the feature choice: at each node, when choosing a feature to use to split, if n features is available, pick a random subset of k < n(usually k = n k = \sqrt{n} k=n ) features and alow the algorithm to only choose from that subset of features
相关推荐
gaosushexiangji42 分钟前
利用sCMOS科学相机测量激光散射强度
大数据·人工智能·数码相机·计算机视觉
ai小鬼头2 小时前
AIStarter新版重磅来袭!永久订阅限时福利抢先看
人工智能·开源·github
说私域2 小时前
从品牌附庸到自我表达:定制开发开源AI智能名片S2B2C商城小程序赋能下的营销变革
人工智能·小程序
飞哥数智坊3 小时前
新版定价不够用,Cursor如何退回旧版定价
人工智能·cursor
12点一刻3 小时前
搭建自动化工作流:探寻解放双手的有效方案(2)
运维·人工智能·自动化·deepseek
未来之窗软件服务3 小时前
东方仙盟AI数据中间件使用教程:开启数据交互与自动化应用新时代——仙盟创梦IDE
运维·人工智能·自动化·仙盟创梦ide·东方仙盟·阿雪技术观
JNU freshman4 小时前
计算机视觉速成 之 概述
人工智能·计算机视觉
说私域5 小时前
基于开源AI大模型AI智能名片S2B2C商城小程序源码的私域流量新生态构建
人工智能·开源
HollowKnightZ5 小时前
目标姿态估计综述:Deep Learning-Based Object Pose Estimation: A Comprehensive Survey
人工智能·深度学习