Variance, and why diversification works

The mean tells you where a bet lands on average. Variance tells you how far it lands from there, and in trading that is the difference between a strategy you can run and one that blows up before its edge shows.

Definition

Variance is the expected squared distance from the mean, and the standard deviation $\sigma$ is its square root, in the same units as $X$:

The second form comes from expanding the square and using linearity. For a fair die, $E[X^2] = 91/6$ and $E[X]^2 = 12.25$, so $\operatorname{Var}(X) = 35/12 \approx 2.92$ and $\sigma \approx 1.71$.

How variance scales and adds

Shifting a variable doesn’t change its spread; scaling it scales the spread:

For sums, there is a cross term:

Only when $X$ and $Y$ are uncorrelated does it reduce to $\operatorname{Var}(X) + \operatorname{Var}(Y)$. The original states that version without the condition; it is the most common slip in interviews, so say “if independent” out loud.

Standard deviations are arrows

Write $\operatorname{Cov}(X,Y) = \rho\,\sigma_X\sigma_Y$ and the formula becomes $\sigma_{X+Y}^2 = \sigma_X^2 + \sigma_Y^2 + 2\rho\sigma_X\sigma_Y$. That is the law of cosines. Draw $\sigma_X$ and $\sigma_Y$ as arrows with the angle between them set so that $\cos(\text{angle}) = \rho$, and the volatility of the sum is the length of the resultant arrow.

This picture also explains the three-correlations puzzle in lesson 5: three arrows can’t have arbitrary pairwise angles.

The √n rule

If $X_1, \dots, X_n$ are independent with variance $\sigma^2$ each,

Averaging $n$ independent things shrinks the noise by $\sqrt n$, not $n$. Four times the data halves the error. If every pair has correlation $\rho$ instead, $\operatorname{Var}(\bar X) = \sigma^2\big(\tfrac1n + (1-\tfrac1n)\rho\big)$, which never falls below $\rho\sigma^2$.

In code

import numpy as np
rng = np.random.default_rng(2)

sx, sy, rho = 3.0, 2.0, 0.3
print(np.sqrt(sx**2 + sy**2 + 2 * rho * sx * sy))   # sd of X + Y: 4.07
print(np.sqrt(sx**2 + sy**2 - 2 * rho * sx * sy))   # sd of the hedge X - Y: 3.07

# Averages of n draws shrink like 1/sqrt(n)
for n in (1, 4, 16, 64):
    means = rng.standard_normal((100_000, n)).mean(axis=1)
    print(n, round(means.std(), 3), round(1 / np.sqrt(n), 3))