机器学习(六) — 评估模型

Evaluate model

1 test set

  1. split the training set into training set and a test set
  2. the test set is used to evaluate the model

1. linear regression

compute test error

J t e s t ( w ⃗ , b ) = 1 2 m t e s t ∑ i = 1 m t e s t [ ( f ( x t e s t ( i ) ) − y t e s t ( i ) ) 2 ] J_{test}(\vec w, b) = \frac{1}{2m_{test}}\sum_{i=1}^{m_{test}} \left [ (f(x_{test}^{(i)}) - y_{test}^{(i)})^2 \right ] Jtest(w ,b)=2mtest1i=1∑mtest[(f(xtest(i))−ytest(i))2]

2. classification regression

compute test error

J t e s t ( w ⃗ , b ) = − 1 m t e s t ∑ i = 1 m t e s t [ y t e s t ( i ) l o g ( f ( x t e s t ( i ) ) ) + ( 1 − y t e s t ( i ) ) l o g ( 1 − f ( x t e s t ( i ) ) ] J_{test}(\vec w, b) = -\frac{1}{m_{test}}\sum_{i=1}^{m_{test}} \left [ y_{test}^{(i)}log(f(x_{test}^{(i)})) + (1 - y_{test}^{(i)})log(1 - f(x_{test}^{(i)}) \right ] Jtest(w ,b)=−mtest1i=1∑mtest[ytest(i)log(f(xtest(i)))+(1−ytest(i))log(1−f(xtest(i))]

2 cross-validation set

  1. split the training set into training set, cross-validation set and test set
  2. the cross-validation set is used to automatically choose the better model, and the test set is used to evaluate the model that chosed

3 bias and variance

  1. high bias: J t r a i n J_{train} Jtrain and J c v J_{cv} Jcv is both high
  2. high variance: J t r a i n J_{train} Jtrain is low, but J c v J_{cv} Jcv is high
  1. if high bias: get more training set is helpless
  2. if high variance: get more training set is helpful

4 regularization

  1. if λ \lambda λ is too small, it will lead to overfitting(high variance)
  2. if λ \lambda λ is too large, it will lead to underfitting(high bias)

5 method

  1. fix high variance:
    • get more training set
    • try smaller set of features
    • reduce some of the higher-order terms
    • increase λ \lambda λ
  2. fix high bias:
    • get more addtional features
    • add polynomial features
    • decrease λ \lambda λ

6 neural network and bias variance

  1. a bigger network means a more complex model, so it will solve the high bias
  2. more data is helpful to solve high variance
  1. it turns out that a bigger(may be overfitting) and well regularized neural network is better than a small neural network
相关推荐
Jamence4 分钟前
多模态大语言模型arxiv论文略读(157)
论文阅读·人工智能·语言模型·自然语言处理·论文笔记
DogDaoDao14 分钟前
Rembg开源项目全面解析:从原理到实践应用
人工智能·深度学习·开源·github·图像分割·背景检测·rembg
汀、人工智能27 分钟前
AI-Compass LLM训练框架生态:整合ms-swift、Unsloth、Megatron-LM等核心框架,涵盖全参数/PEFT训练与分布式优化
人工智能·分布式·sft·swift·大模型训练
ATM00628 分钟前
开源AI Agent开发平台Dify源码剖析系列(二)
人工智能·开源·dify·源码剖析
ATM0062 小时前
人机协作系列(四)AI编程的下一个范式革命——看Factory AI如何重构软件工程?
人工智能·大模型·agent·人机协作·人机协同
读创商闻3 小时前
极狐GitLab CEO 柳钢——极狐 GitLab 打造中国企业专属 AI 编程平台,引领编程新潮流
人工智能·gitlab
kailp3 小时前
语言模型玩转3D生成:LLaMA-Mesh开源项目
人工智能·3d·ai·语言模型·llama·gpu算力
marteker3 小时前
弗兰肯斯坦式的人工智能与GTM策略的崩溃
人工智能·搜索引擎
无心水3 小时前
大语言模型零样本情感分析实战:无需机器学习训练,96%准确率实现指南
人工智能·机器学习·语言模型
来自于狂人3 小时前
AI大模型训练的云原生实践:如何用Kubernetes指挥千卡集群?
人工智能·云原生·kubernetes