Joint distributions, covariance and correlation

Single random variables price single bets. Trading is mostly about pairs and portfolios: a stock and its hedge, a signal and next-day returns. For that you need to describe how two random things move together.

Joint and marginal distributions

Two variables have a joint distribution, $p_{X,Y}(x,y) = P(X=x, Y=y)$ in the discrete case, or a joint density with

f_{X,Y}(x,y) = \frac{\partial^2 F_{X,Y}(x,y)}{\partial x\,\partial y}, \qquad F_{X,Y}(x,y) = P(X\le x, Y\le y).

To get back the distribution of one variable alone (its marginal), sum or integrate out the other: $p_X(x) = \sum_y p_{X,Y}(x,y)$ and $f_X(x) = \int f_{X,Y}(x,y)\,dy$ .

$X$ and $Y$ are independent exactly when the joint factorises: $f_{X,Y}(x,y) = f_X(x)\,f_Y(y)$ for every $x$ and $y$ .

Covariance

\operatorname{Cov}(X,Y) = E\big[(X-E[X])(Y-E[Y])\big] = E[XY] - E[X]\,E[Y].

“Expectation of the product minus the product of the expectations.” Positive when the two tend to be above their means together. Useful facts:

$\operatorname{Cov}(X,X) = \operatorname{Var}(X)$ .
It is bilinear: $\operatorname{Cov}(aX_1 + bX_2, Y) = a\operatorname{Cov}(X_1,Y) + b\operatorname{Cov}(X_2,Y)$ .
Independent ⇒ $\operatorname{Cov} = 0$ . The converse is false.

Correlation

Covariance depends on units (dollars times shares). Correlation rescales it to $[-1, 1]$ :

\rho(X,Y) = \frac{\operatorname{Cov}(X,Y)}{\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}} .

$\rho = \pm1$ means one variable is an exact linear function of the other. $\rho = 0$ means no linear relationship, which is not the same as no relationship.

Figure 5.1The joint density as a landscape

The bivariate normal density for two standardised variables. Slide ρ and the hill swings onto the diagonal; the marginal curves on the back walls never move. The dots on the floor are 400 draws, and the orange ring holds about 68% of them.

Figure 5.2Correlation scatter

Slide ρ and watch the cloud tilt; the orange line is the least-squares fit, whose slope is Cov(X, Y)/Var(X). Then pick “Y = X²”: Y is completely determined by X, yet the correlation is about zero. Three outliers show how fragile a sample correlation can be.

The three-correlations puzzle

Jane Street (reported). Corr(X, Y) = 0.9 and Corr(Y, Z) = 0.8. What are the tightest bounds on Corr(X, Z)?

Any valid correlation matrix is positive semidefinite: all its eigenvalues are non-negative, so its determinant (their product) is too. With $c = \operatorname{Corr}(X,Z)$ ,

\det\begin{pmatrix}1 & 0.9 & c\\ 0.9 & 1 & 0.8\\ c & 0.8 & 1\end{pmatrix} = -c^2 + 1.44c - 0.45 \;\ge\; 0 \quad\Longrightarrow\quad 0.458 \le c \le 0.982 .

In general $c \in \rho_{XY}\rho_{YZ} \pm \sqrt{(1-\rho_{XY}^2)(1-\rho_{YZ}^2)}$ . Geometrically (lesson 3), correlations are cosines of angles between arrows: X is 25.8° from Y and Z is 36.9° from Y, so the angle between X and Z is between 11.0° and 62.7°.

Figure 5.3Three-correlations bound

The curve is the determinant of the correlation matrix as a function of the unknown correlation. Only the green band, where it is non-negative, is achievable. Try your own values for the two known correlations.

In code

import numpy as np

C = lambda c: np.array([[1, 0.9, c], [0.9, 1, 0.8], [c, 0.8, 1]])
valid = [c for c in np.linspace(-1, 1, 20001) if np.linalg.eigvalsh(C(c)).min() >= -1e-12]
print(round(min(valid), 3), round(max(valid), 3))     # 0.458 0.982

# Dependent but uncorrelated
rng = np.random.default_rng(3)
x = rng.standard_normal(100_000)
y = x ** 2
print(np.corrcoef(x, y)[0, 1])                        # ≈ 0

Check yourself

X is uniform on [−1, 1] and Y = |X|. Which is true?