Bias-variance decomposition

Mathematical statement

For every theta supply a complete normalized finite sampling law. Under that same law separately compute the mean, bias=E_theta[T]−theta, variance=E_theta[(T−E_theta[T])^2] and MSE=E_theta[(T−theta)^2]. Expanding T−theta=(T−E_theta[T])+bias gives a cross term that vanishes because E_theta[T−E_theta[T]]=0. Thus MSE=variance+bias^2 pointwise in theta. Do not pool different theta laws. Unbiasedness gives MSE=variance, not zero MSE or consistency. One observed squared error is not the definition of expected loss, even if its numerical value coincides with MSE.

Representative formula (exact TeX)

M(T,t)=V(T,t)+B(T,t)^2

Assumptions

• Theta, listed estimator values and probabilities are exact rationals in a complete normalized law.
• This slice uses scalar estimators with finite sampling laws.

Symbol scope

• M — mean squared error
• T — scalar estimator
• t — target parameter
• V — estimator variance
• B — estimator bias

Intuition, definition, and misconceptions

Compute all four quantities under the same complete law.

Cross-term cancellation requires the explicit centering identity.

The biased/unbiased boundary is not a zero-error claim.

A single realized squared error is not MSE, and an unbiased estimator can still have positive variance.

Adopted derivation guide

The original MKO is retained. The guide has its own append-only revision and verification boundaries, not a new Lean proof.

Prerequisites

• Mean squared error of an estimator — Use its theta-specific definition.
• Estimator bias — Use the bias definition.
• Estimator variance — Use variance under the same law.

Verification boundary

Check each theta separately. Finite tables do not establish consistency or asymptotic optimality.

not_formalized

This does not mean the content is false; it means no replayable computational or formal evidence is registered yet.

Next obligation

Add finite-model risks and estimator comparisons.

Same-source structured data

This page and the interactive view use the same object and existing language edition; this adds no proof or human review.