Maximum_Likelihood

Statistics with Prof. Liu Sept 6, 2024

Statistics has two streams: frequenties, and Bayes.

Maximum likelihood is frequenties method.

Likelihood function is powerful, which contains all information for the data. We don't need others, just this function.

Likelihood function is the joint probability of all data. Likelihood 就是data的概率。所有data的信息全在这个likelihood finction里面了!

We assume data are iid. In statistics we say iid is "random sample". It means, e.g. 每次抽球的概率一样。

That Joint probability is just the product of all P(data | parameter)

It's data's probability! Not parameters' probability.

例:两个盒子,一个有5个黑球5个白球,另一个有9个黑球1个白球。现抽4次,每次放回地抽1个球。4个都是黑球。问最可能从哪个盒子抽的。

For box1, if 4 times, P(4 black) = P(black)^4 = 0.5^4

For box2, if 4 times, P(4 black) = P(black)^4 = 0.9^4 that's why iid and we make product.

The latter probability is larger, so we choose box2.

The parameters for each box: box1, binomial, n=4, p=0.5. box2, n=4, p=0.9

例,无穷多个盒子,它们有黑球的比例是从0到1不等。抽4次,4个都是黑球。问从哪个盒子抽的。

Now the parameters, n=4, but don't know p. Want to know p, once know p, we then know which box.

**We still choose the box with the highest p. We choose the max P(data given p) ie P(data given box). **

But what is the probability that box2 is what I have done? What is the probability for box1?

But these numbers are not the probabilities for the two boxes! It's more intuitive to make decisions based on their probabilities, such as the prob of rain 40% and not rain 60%.

It has logics. We are choosing the parameter which can make the data to be most likely to stand.

Likelihood is just the probability, the probability of data. 0 to 1.

**应用到科学方法论,We can measure the distance of a theory to the real world data, ie, to examine a theory is good or bad, using likelihood. **


Statistics do inference: estimation and prediction.

Estimation has two categories: 1. Assume we know the population distribution, we just estimate its parameters. 2. We don't even know the population distribution.

After that, if we use our model to fit new data, then it's prediction.

**Prediction error is larger than estimation error. ** Estimatiin error is just RSS, the sum of square residuals. But for prediction, a new dataset will introduce new noise, and plus the model's original RSS.


In logistics regression, and linear regression, and linear discriminate analysis, the conditional class probabilities sum up to 1, and thus is posterior probability P(parameter given data). Just compare them directly. We can use Bayes optimal classifier. It's not related to likelihood.

相关推荐
小七在进步7 小时前
string的实现(1)
c++·笔记·visual studio
保利威视频SaaS小张10 小时前
企业直播接入笔记 12|企业内训直播怎么做签到和学时统计,防挂机这件事比想象中难
java·前端·数据库·笔记·视频云·polvy·polyv
lcj251112 小时前
[C++] C++11
开发语言·c++·笔记
一条破秋裤13 小时前
【1117】【内存资源管控】【笔记】
android·笔记
qingwufeiyang_53015 小时前
Docker学习笔记2
笔记·学习
Z21-X16 小时前
云原生运维学习笔记:从操作系统底层到 Kubernetes 集群的深度实践
运维·笔记·学习·云原生·kubernetes
星恒随风16 小时前
MySQL 从零入门(七):索引原理与 B+ 树详解——从磁盘 I/O 到 InnoDB 聚簇索引
linux·服务器·数据库·笔记·mysql
希和20271 天前
江科大 51 单片机笔记:DS18B20 温度传感器 + LCD1602 显示代码详解
笔记·单片机·mongodb
Sakura_j_h_1 天前
MCP SDK 2.0 实战:用 MCPServer 搭本地笔记检索服务
开发语言·笔记·python·qt·ai工具·mcp·mcpserver
前进的李工1 天前
Agent 学习笔记 15:Agent 怎样响应事件与中途打断?
人工智能·笔记·agent·openclaw·工具设计·gpt-6 astra·原生异步