Logistic Regression And Regularization

Prior to it , we learn a classic regression algorithm.Now I will show you a case of another important superviser learning:Logistic regression.

Please heeding! 'Logistic regression' is not a regression though its name contains 'regression'. It is a specifical algorithm to transform the expression of binary linear regression. you can get a estimation percentage instead of unlogical regression result.Because, It belongs to classification algorithms.

Maybe this santences is also unintelligible,now let's to describe it step by step.

Why we don't use the linear Regression to solve Binary Regression Problems?

let's see this situation:you are a tumor specialized doctor。In your medical living,you have token p part in operation many times and go through mang size data。From experiment,you can easily judge whether the tumor is benign or maglignant by its size。

Well,you get a binary target value graph when you put the data to a graph

(About axis: "yes or no" denotes as Y ,the size of tumor denotes as X)

In this graph,yep,we can use linear model to regress and that is no problem.

In general,we should set a threshold(communly 0.5 in this binary question).If the prediction value exceeds threshold,declares malignant。If the prediction value is less then threshold,declares benign。

So,get a linear model

When I get a newly data and perdict a result "0.7",I can annource that this tumor is malignant......

But in fact,this tumor is benign.

This is a unlogical error ! We can say "a malignant tumor with 70% probability",but not "0.7 tumor".We want to get a more scientifical and reasonable result,not just a calm num。

Based on this,logistic regression can transform the pure number result to a estimation result。"It has 70% probability to assert this tumor is malignant。" This sentance is logical,in my view。

How to start logistic regression?

Before starting,I must tell you Logistic regression is not regression but classification algorithm.It can be regard as a specialized method what is used to logically solve binary linear regression.

In this method , we induce a specialized function called 'sigmoid'

In this situcation,z=wx+b

so,we can get and a new graph.

sigmoid function is a very useful function,the curve line is naturally divdied into two parts by the point 0 and threshold 0.5.And the output is limited in 0 to 1.

The output of this model means the probability of "yes"(denote as f(x)=p(1),you must (记住) it!)

For example, If f(x)=0.99,we can say "It has 99% probability to judge a malignant tumor".Even if the yi(real situation),we still consider this prediction estimation reasonably.

Loss function:

In regression , we evaluate fitting dregees in one point by error function (yi'-yi)

For logistic regression ,error function is not a suitable function to count error or loss.There is a lot of local minima that you may get stuck.

So we induce a new notion : loss Function,to evaluate the sacle of loss in one point

(Notice: In the behind, f(x) is equals to g(z))

why we define this odd function? The answer is more reasonable

I set a example: when the real situation is "yes",but the f(x)->0

That means "We judge that it is absolutly a benign tumor,however, it is no",and Loss will pursue 1 according to this function, signfiting this fit operation is uncorrect.If you are interested in other situations,you can draw a graph or try in open lab.

If the prediction value settles near to real result, the loss will bacome small and reasonable.

Cost Function:

In linear regression ,we use error function to accumlate a cost value.Now we change it into loss function.

As the fact, The definition of loss cost function is equals to error function

so ,we can unfold it:

conventent to count? Maybe......

Gradient descend:

only need "simultaneous update"

the scale of fit

so,there are there situation

1.Underfit: don't fit the training set wall

2.no name,no error

3.overfit:extremely well.....we don't think it is a good fitting

we focus on the 3-rd situation:

overtraining make a well but high varience。

we have three method to fix the overfit:

(1)collect more data to train a reasonable model

(2)only select main features(substract some feature)

(3)reduce the size of parameter

(3) is a usual method consist of some operation such as regularization, and penlize

regularization, and penlize

To make a model low variance, we must reduce some parameter.

The operation of reducing is "penlize", The entire process is regularization.

In fact,the method we used is called "L2 regularization".Its target is reducing the size of some parameter. "Penlize" is a technolegical term meaning a reduce paramter.

For example,If Iwant to reduce parameter w1,we can do this in iterating process.At the end of loss or error function,we add a item ""

so in iterating function:, this make a effensive cut in w1.

But in usual,we don't know what para should be penlized,so we decide to pandize all parameters.

At the end of loss or errorfunction,we add a item

to maintain a proportional scale for each parameter."

相关推荐
云知谷4 小时前
【C++基本功】C++适合做什么,哪些领域适合哪些领域不适合?
c语言·开发语言·c++·人工智能·团队开发
rit84324994 小时前
基于MATLAB实现基于距离的离群点检测算法
人工智能·算法·matlab
初学小刘5 小时前
深度学习:从图片数据到模型训练(十分类)
人工智能·深度学习
递归不收敛6 小时前
大语言模型(LLM)入门笔记:嵌入向量与位置信息
人工智能·笔记·语言模型
之墨_6 小时前
【大语言模型】—— 自注意力机制及其变体(交叉注意力、因果注意力、多头注意力)的代码实现
人工智能·语言模型·自然语言处理
从孑开始7 小时前
ManySpeech.MoonshineAsr 使用指南
人工智能·ai·c#·.net·私有化部署·语音识别·onnx·asr·moonshine
涛涛讲AI7 小时前
一段音频多段字幕,让音频能够流畅自然对应字幕 AI生成视频,扣子生成剪映视频草稿
人工智能·音视频·语音识别
可触的未来,发芽的智生7 小时前
新奇特:黑猫警长的纳米世界,忆阻器与神经网络的智慧
javascript·人工智能·python·神经网络·架构
悟乙己8 小时前
MLops | 基于AWS Lambda 架构构建强大的机器学习(ML)血缘关系
机器学习·架构·aws
WWZZ20258 小时前
快速上手大模型:机器学习2(一元线性回归、代价函数、梯度下降法)
人工智能·算法·机器学习·计算机视觉·机器人·大模型·slam