机器学习(六) — 评估模型

Evaluate model

1 test set

  1. split the training set into training set and a test set
  2. the test set is used to evaluate the model

1. linear regression

compute test error

J t e s t ( w ⃗ , b ) = 1 2 m t e s t ∑ i = 1 m t e s t [ ( f ( x t e s t ( i ) ) − y t e s t ( i ) ) 2 ] J_{test}(\vec w, b) = \frac{1}{2m_{test}}\sum_{i=1}^{m_{test}} \left [ (f(x_{test}^{(i)}) - y_{test}^{(i)})^2 \right ] Jtest(w ,b)=2mtest1i=1∑mtest[(f(xtest(i))−ytest(i))2]

2. classification regression

compute test error

J t e s t ( w ⃗ , b ) = − 1 m t e s t ∑ i = 1 m t e s t [ y t e s t ( i ) l o g ( f ( x t e s t ( i ) ) ) + ( 1 − y t e s t ( i ) ) l o g ( 1 − f ( x t e s t ( i ) ) ] J_{test}(\vec w, b) = -\frac{1}{m_{test}}\sum_{i=1}^{m_{test}} \left [ y_{test}^{(i)}log(f(x_{test}^{(i)})) + (1 - y_{test}^{(i)})log(1 - f(x_{test}^{(i)}) \right ] Jtest(w ,b)=−mtest1i=1∑mtest[ytest(i)log(f(xtest(i)))+(1−ytest(i))log(1−f(xtest(i))]

2 cross-validation set

  1. split the training set into training set, cross-validation set and test set
  2. the cross-validation set is used to automatically choose the better model, and the test set is used to evaluate the model that chosed

3 bias and variance

  1. high bias: J t r a i n J_{train} Jtrain and J c v J_{cv} Jcv is both high
  2. high variance: J t r a i n J_{train} Jtrain is low, but J c v J_{cv} Jcv is high
  1. if high bias: get more training set is helpless
  2. if high variance: get more training set is helpful

4 regularization

  1. if λ \lambda λ is too small, it will lead to overfitting(high variance)
  2. if λ \lambda λ is too large, it will lead to underfitting(high bias)

5 method

  1. fix high variance:
    • get more training set
    • try smaller set of features
    • reduce some of the higher-order terms
    • increase λ \lambda λ
  2. fix high bias:
    • get more addtional features
    • add polynomial features
    • decrease λ \lambda λ

6 neural network and bias variance

  1. a bigger network means a more complex model, so it will solve the high bias
  2. more data is helpful to solve high variance
  1. it turns out that a bigger(may be overfitting) and well regularized neural network is better than a small neural network
相关推荐
MARS_AI_3 小时前
云蝠智能 Voice Agent 落地展会邀约场景:重构会展行业的智能交互范式
人工智能·自然语言处理·重构·交互·语音识别·信息与通信
weixin_422456444 小时前
第N7周:调用Gensim库训练Word2Vec模型
人工智能·机器学习·word2vec
HuggingFace7 小时前
Hugging Face 开源机器人 Reachy Mini 开启预定
人工智能
企企通采购云平台8 小时前
「天元宠物」×企企通,加速数智化升级,“链”接萌宠消费新蓝海
大数据·人工智能·宠物
超级小忍8 小时前
Spring AI ETL Pipeline使用指南
人工智能·spring
张较瘦_8 小时前
[论文阅读] 人工智能 | 读懂Meta-Fair:让LLM摆脱偏见的自动化测试新方法
论文阅读·人工智能
巴伦是只猫9 小时前
【机器学习笔记 Ⅲ】4 特征选择
人工智能·笔记·机器学习
好心的小明9 小时前
【王树森推荐系统】召回11:地理位置召回、作者召回、缓存召回
人工智能·缓存·推荐系统·推荐算法
lishaoan779 小时前
使用tensorflow的线性回归的例子(十二)
人工智能·tensorflow·线性回归·戴明回归
Danceful_YJ10 小时前
4.权重衰减(weight decay)
python·深度学习·机器学习