Mean squared error: bias² plus variance
The average squared error of an estimate splits into two parts: systematic error (bias, squared) and noise (variance). Accepting a little bias can remove a lot of noise.
01The formula
- an estimator: a rule that turns data into a guess
- the true value being estimated, here the mean μ
- how far the estimator is off on average
- how much the estimator moves from sample to sample
- shrinkage: how far the sample mean is pulled towards zero
- the sample mean
- the true mean
- the volatility of one observation and the number of observations
02See it move
03Where it is used
- Quant researcherShrinks noisy expected-return estimates towards zero or a prior before building portfolios.
- Portfolio managerUses shrunk covariance matrices, which trade a little bias for much more stable weights.
- Risk managerJudges a volatility estimator by its total error, not just by whether it is unbiased.
04Why traders use it
Unbiased estimates of mean returns are so noisy that they make poor inputs. Shrinking them is one of the most reliable improvements in practice, and this split shows why.
05Worked example
With a true mean of 0.05%, volatility 1.0% and 250 days, shrinking the sample mean by 30% gives an MSE of 21.8 bp², made of bias² 2.3 and variance 19.6. The best shrinkage here is 62%.
06How it is built
- Add and subtract : .
- Square and take expectations; the cross term has mean zero, leaving .
- For : the bias is and the variance .
- Minimising over gives : shrink more when the data are noisy.
07The common mistake
Insisting on unbiased estimators. An unbiased estimate with huge variance is usually further from the truth than a slightly biased, much steadier one.
08In Python and LaTeX
mu, sigma, n, s = 5.0, 100.0, 250, 0.3 # in basis points bias2 = (s*mu)**2 var = (1 - s)**2 * sigma**2 / n print(round(bias2, 2), round(var, 2), round(bias2 + var, 2))
\operatorname{MSE}(\hat\theta) = E\big[(\hat\theta - \theta)^2\big] = \operatorname{Bias}^2 + \operatorname{Var}, \qquad \hat\theta = (1 - s)\bar{x}:\ \ \operatorname{MSE} = s^2\mu^2 + (1 - s)^2\frac{\sigma^2}{n}09Where it fits
Rather not code the maths yourself? Describe your trading idea and let AI build the bot.
Build a bot with AgenKitThe worked example and the model run the same code, so every number matches. Models simplify markets on purpose. This is education, not financial advice.