What Is Machine Learning · L4 Foundations

The mathematics of bias-variance decomposition, VC dimension intuition, and why more parameters doesn't always mean more overfitting.

Bias²error from oversimplified model

Varianceerror from sensitivity to training sample

Irreducible noisecannot be removed by any model

A decomposition of total expected test error into three additive components.

For a regression problem with true function $f (x)$ , noisy observations $y = f (x) + ϵ$ with $E [ϵ] = 0, Var (ϵ) = σ^{2}$ , and a model $f^$ trained on a random sample, the expected squared error at a point $x$ decomposes exactly as: