机器学习(七) — 决策树

model 4 --- decision tree

1 decision tree

1. component

usage: classification

  1. root node
  2. decision node

2. choose feature on each node

maximize purity (minimize inpurity)

3. stop splitting

  1. a node is 100% on class
  2. splitting a node will result in the tree exceeding a maximum depth
  3. improvement in purity score are below a threshold
  4. number of examples in a node is below a threshold

2 meature of impurity

use entropy( H H H) as a meature of impurity

H ( p ) = − p l o g 2 ( p ) − ( 1 − p ) l o g 2 ( 1 − p ) n o t e : 0 l o g 0 = 0 H(p) = -plog_2(p) - (1-p)log_2(1-p)\\ note: 0log0 = 0 H(p)=−plog2(p)−(1−p)log2(1−p)note:0log0=0

3 information gain

1. definition

i n f o m a t i o n _ g a i n = H ( p r o o t ) − ( w l e f t H ( p l e f t ) + w r i g h t H ( p r i g h t ) ) infomation\_gain = H(p^{root}) - (w^{left}H(p^{left}) + w^{right}H(p^{right})) infomation_gain=H(proot)−(wleftH(pleft)+wrightH(pright))

2. usage

  1. meature the reduction in entropy
  2. a signal of stopping splitting

3. continuous

find the threshold that has the most infomation gain

4 random forest

  1. generating a tree sample
复制代码
given training set of size m
for b = 1 to B:
	use sampling with replacement to create a new training set of size m
	train a decision tree on the training set
  1. randomizing the feature choice: at each node, when choosing a feature to use to split, if n features is available, pick a random subset of k < n(usually k = n k = \sqrt{n} k=n ) features and alow the algorithm to only choose from that subset of features
相关推荐
后端小肥肠9 分钟前
公众号里流量很夯的漫画赛道:趣味漫画创作全拆解
人工智能·aigc·agent
程序油菜花30 分钟前
微软 ONNX Runtime 仓库突然 404:你的项目,可能并不真正属于你
人工智能
锋行天下42 分钟前
LangGraph 对接 MCP 实战指南:从本地开发到线上部署与模型防坑
前端·人工智能·后端
汤姆宝库1 小时前
解码采样策略的实现:temperature、top-p、min-p与重复惩罚
人工智能
鶴哥只手遮天1 小时前
《WeKnora 深度技术解析》第 5 章 数据底座:多租户模型、仓储与迁移
人工智能
guuguo1 小时前
【做个开源软件】Mac 上 node 进程莫名吃掉 6G 内存?给 AI / MCP 做了个深度归因和安全回收工具
人工智能·开源·github
双鱼大猫2 小时前
opencode 吃掉 100G 空间:Agent 会话数据要分级清理
人工智能·agent·ai编程
用户5833339816682 小时前
RPA 流程的异常处理设计:重试、降级与告警,生产环境实战指南
人工智能
sanxing三省3 小时前
DeepSeek Harness 到底是什么?标题里最重的词其实不是「DeepSeek」
前端·人工智能·deepseek
合尘猫4 小时前
用 6GB 显卡搭短视频产线:MiniMax H3 + ffmpeg 全流程实录(55 秒片 / 40 分钟)
人工智能·音视频