Bias-variance derivation with the cross term visible

Goal and scope

Prove the decomposition symbol by symbol in one law and check both theta values.

Finite scalar laws, kept separate at each theta.

Givens and symbols

• work

• G1 — Theta={1/4,3/4}, Omega={00,01,10,11}; P_(1/4)=(9,3,3,1)/16 and P_(3/4)=(1,3,3,9)/16.
• G2 — U(x)=(x1+x2)/4 has total table(0,1/4,1/4,1/2) in Omega order, with target g(theta)=theta.

• θ — fixed unknown parameter
• U — one total statistic
• m,b — same-law mean and bias

Rules and sources

• mko_definition · cited — For every theta supply a complete normalized finite sampling law. Under that same law separately compute the mean, bias=E_theta[T]−theta, variance=E_theta[(T−E_theta[T])^2] and MSE=E_theta[(T−theta)^2]. Expanding T−theta=(T−E_theta[T])+bias gives a cross term that vanishes because E_theta[T−E_theta[T]]=0. Thus MSE=variance+bias^2 pointwise in theta. Do not pool different theta laws. Unbiasedness gives MSE=variance, not zero MSE or consistency. One observed squared error is not the definition of expected loss, even if its numerical value coincides with MSE. · mko-bias-variance-decomposition
• work · prerequisite — Linearity, constant expectations under normalized laws and square expansion; at fixed theta, m=EthetaU, bias=m-theta, variance=Etheta[(U-m)²] and MSE=Etheta[(U-theta)²]. The decomposition itself is not assumed.

Main derivation

\theta\in\Theta\text{ fixed},\quad m=E_\theta U

b=m-\theta,\quad U-\theta=(U-m)+b

U-\theta=(U-m)+b

(U-\theta)^2=(U-m)^2+2b(U-m)+b^2

E_\theta[U-m]=E_\theta U-m

E_\theta[U-m]=0

E_\theta[2b(U-m)]

2bE_\theta[U-m]=0

E_\theta[(U-\theta)^2]

E_\theta[(U-m)^2]+0+b^2

\operatorname{MSE}_\theta(U)=E_\theta[(U-m)^2]+b^2

\operatorname{MSE}_\theta(U)=\operatorname{Var}_\theta(U)+b(\theta)^2

\theta=1/4,\quad U\in\{0,1/4,1/2\},\quad P_{1/4}^U=(9,6,1)/16

m=0(9/16)+(1/4)(6/16)+(1/2)(1/16)=1/8,\quad b=1/8-1/4=-1/8

v=\sum_uP_{1/4}^U(u)(u-1/8)^2,\quad M=\sum_uP_{1/4}^U(u)(u-1/4)^2

v=\frac{9(1/8)^2+6(1/8)^2+(3/8)^2}{16}=3/128;\quad M=\frac{9(1/4)^2+6(0)^2+(1/4)^2}{16}=5/128

\theta=1/4:\quad M=5/128,\ b=-1/8,\ v=3/128

5/128=3/128+(-1/8)^2

\theta=3/4,\quad P_{3/4}^U=(1,6,9)/16

m=0(1/16)+(1/4)(6/16)+(1/2)(9/16)=3/8,\quad b=3/8-3/4=-3/8

v=\sum_uP_{3/4}^U(u)(u-3/8)^2,\quad M=\sum_uP_{3/4}^U(u)(u-3/4)^2

v=\frac{(3/8)^2+6(1/8)^2+9(1/8)^2}{16}=3/128;\quad M=\frac{(3/4)^2+6(1/2)^2+9(1/4)^2}{16}=21/128

\theta=3/4:\quad M=21/128,\ b=-3/8,\ v=3/128

21/128=3/128+(-3/8)^2

Substeps and lemmas

No substeps in this guide.

Reverse audits

• S12 · premise_audit · G1, G2, S11, work — Unbiasedness only removes squared bias, not variance.
• S11 · premise_audit · G1, G2, S10, work — The three direct MSE contributions are 9/256,24/256,9/256.
• S10 · premise_audit · G1, G2, S09, work — The final support mass under this law is 9/16.
• S09 · premise_audit · G1, G2, S08, work — Square bias first; negative bias is not subtracted.
• S08 · premise_audit · G1, G2, S07, work — The variance center is 1/8 while the MSE target is 1/4; they differ.
• S07 · premise_audit · G1, G2, S06, work — Pair the three support values with their three masses in the same order.
• S06 · premise_audit · G1, G2, S05, work — Theta was arbitrary, so the identity holds separately at every model parameter.
• S05 · premise_audit · G1, G2, S04, work — Normalization gives Etheta[b²]=b².
• S04 · premise_audit · G1, G2, S03, work — No independence is required here.
• S03 · premise_audit · G1, G2, S02, work — Signs alone do not give mean zero; the definition of m does.
• S02 · premise_audit · G1, G2, S01, work — The factor2 is required; b is not a random observed value.
• S01 · premise_audit · G1, G2, work — Both m and b belong to this Ptheta, not a mixture.

Error controls and negative controls

Visible cross term

Retain2b(U-m) before eliminating it by centered expectation zero.

Direct MSE audit

Compare both direct loss sums with 3/128 plus squared bias.

• The cross term vanishes only by independence. → reject — b is constant and Etheta[U-m]=0 suffices.
• Unbiasedness makes MSE zero. → reject — It only gives MSE=variance; an unbiased Bernoulli observation has positive variance.
• MSE=variance+bias. → reject — Expansion gives squared bias; unsquared bias would even give a negative value in the first law.

Result and limitations

MSEtheta=Vartheta+bias(theta)²; direct values are 5/128 and 21/128 at the two theta values.

Verification and source binding

6 declared checks for this guide; only the listed expressions are covered.

• source commit: b1b2f1dcf8c50d67a93fe52d95f8eef7b3abba9f
• adopted source SHA-256: 684c90c6e4cc67e67bac21d84d6391f4ac7df58ae8a555e711b2c67724a97dc5
• original source SHA-256: 8b7541177d92f5388c355193b85c8de3a5067e1f823493728c074e77a687ec37
• Lean: NOT_RUN · independent AI: SCOPED_LEAD_AND_BILINGUAL_CONTEXT_REVIEW

Same-source full JSON (formulas, ASTs, controls and revision log)