Estimator (Statistic for Machine Learning)

Estimator 估計器

免費完整版：link

An estimator is a mathematical rule, function, or formula used to approximate an unknown population parameter (such as the mean, variance, or proportion) based on sample data. In statistical analysis, estimators are essential because they provide insights about a population when it is impractical or impossible to measure the entire population directly.

Key Concepts of Estimators

1. Population vs. Sample：

• Population Parameter ( θ \theta θ): A fixed but unknown characteristic of a population, like the true mean ( μ \mu μ) or variance ( σ 2 \sigma^2 σ2). 總體的固定但未知的特征
• Esimator估計器( θ ^ \hat{\theta} θ^): A statistic (function of sample data) used to estimate the population parameter. 用於估計總體參數的統計量

2. Notation :符号：

• The true population parameter is denoted by θ \theta θ. 真實總體參數
• The estimator (based on the sample) is denoted by θ ^ \hat{\theta} θ^. 基於樣本的估計量
  • Population mean: μ \mu μ.
  • Sample mean(estimator): x ˉ \bar{x} xˉ.

Type of Estimators

1. Point Estimator: 點估計器

• Provides a single value estimate of a population parameter. 提供总体参数的单值估计。
• Example: The sample mean ( x ˉ \bar{x} xˉ) is a point estimator of the population mean ( μ \mu μ).

2. Interval Estimator : 區間估計器

• Provides a range of values within which the population parameter is likely to lie. 提供总体参数可能位于的值范围
  • Example: Confidence intervals for the mean. 平均值的置信区间

Properties of a Good Estimator

1. Unbiasedness：

• An estimator is unbiased if its expected value equals the true parameter: E θ \^ = θ E\\hat{θ}=θ Eθ\^=θ 如果估计器的期望值等于真实参数，则该估计器是无偏的
  • Example: The sample mean ( x ˉ \bar{x} xˉ) is an unbiased estimator of the population mean ( μ \mu μ)

2. Consistency:

• An estimator is consistent if it converges to the true parameter value as the sample size increases θ ^ → n → ∞ θ \hat{\theta} \xrightarrow{n \to \infty} \theta θ^n→∞ θ 如果随着样本量的增加，估计量收敛到真实参数值，则该估计量是一致的

3. Efficiency:

• An estimator is efficient if it has the smallest possible variance among all unbiased estimators. 如果一个估计量在所有无偏估计量中具有最小的可能方差，则该估计量是有效的

4. Sufficiency 充足性

• An estimator is sufficient if it captures all the information in the sample relevant to the parameter being estimated. 如果估计器捕获了样本中与所估计的参数相关的所有信息，则该估计器就足够了

Examples of Estimators

1. Sample Mean:

• Estimator for the population mean ( μ \mu μ) : 总体平均值的估计量
  μ ^ = x ˉ = 1 n ∑ i = 1 n x i \hat{\mu} = \bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i μ^=xˉ=n1i=1∑nxi

2. Sample Variance

• Biased estimator

  s 2 = 1 n ∑ i = 1 n ( x i − x ˉ ) 2 s^2 = \frac{1}{n} \sum_{i=1}^{n} (x_i - \bar{x})^2 s2=n1i=1∑n(xi−xˉ)2

• Unbiased estimator:

σ ^ 2 = 1 n − 1 ∑ i = 1 n ( x i − x ˉ ) 2 \hat{\sigma}^2 = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2 σ^2=n−11i=1∑n(xi−xˉ)2

3. Sample Proportion:

• Estimator for the population proportion ( p p p): 人口比例的估计量

p ^ = x n \hat{p} = \frac{x}{n} p^=nx

Where x x x is the number of successes and n n n is the sample size.

Evaluation of Estimators

1. Mean Squared Error (MSE) :均方误差

• Combines both bias and variance to evaluate an estimator: 结合偏差和方差来评估估计量

MSE ( θ ^ ) = Var ( θ ^ ) + Bias ( θ \^ ) 2 \text{MSE}(\hat{\theta}) = \text{Var}(\hat{\theta}) + \left\\text{Bias}(\\hat{\\theta})\\right^2 MSE(θ^)=Var(θ^)+Bias(θ\^)2

• Lower MSE indicates a better estimator 较低的 MSE 表示更好的估计量

2. Bias-Variance Tradeoff 偏差方差權衡

• A tradeoff between minimizing bias and variance. In some cases, a slightly biased estimator with lower variance might be preferred (e.g., Ridge Regression).

Estimators in Statistical Inference 统计推断中的估计器

Estimators are widely used in:估计器广泛用于

1. Parameter Estimation

• Estimating population parameters (e.g., mean, variance, correlation).估计总体参数（例如平均值、方差、相关性）

2. Hypothesis Testing :假设檢驗

• Using estimators to calculate test statistics.

3. Machine Learning :机器学习：

• Estimating model parameters to minimize loss functions.估计模型参数以最小化损失函数。

summary

An estimator is a tool for inferring population parameters from sample data. Its quality is determined by properties such as unbiasedness, consistency, and efficiency. Choosing or constructing a good estimator is central to statistical inference, enabling accurate and reliable conclusions about the population.