Conditional probability and Bayes’ theorem

Trading is updating a belief under uncertainty, many times a second. A market maker shows a price, someone trades against it, and the market maker has to ask: given that this just happened, what is the thing actually worth? That question has a precise answer, and it is the most tested idea in quant interviews.

Conditional probability: shrink the world

Two events are rarely unrelated. The chance someone is sick changes once you know they are coughing. For events $A$ and $B$, the probability of $A$ given that $B$ happened is

Read it geometrically. Once you learn that $B$ happened, every outcome outside $B$ is impossible, so $B$ becomes your whole world. $P(A\mid B)$ is the share of that new world where $A$ also happened.

Bayes’ theorem: flip the condition

The intersection can be written two ways, $P(A\mid B)P(B) = P(A\cap B) = P(B\mid A)P(A)$. Divide by $P(B)$ and you get Bayes’ theorem:

Here $H$ is a hypothesis (the cab was blue, the coin is rigged) and $E$ is the evidence you saw. The bottom line is the law of total probability: the evidence can arrive either because $H$ is true or because it isn’t, so you add both routes.

The useful part is that the right-hand side is usually easy to get. “How often does a good witness say blue when the cab is blue?” is a property of the witness. “How common are blue cabs?” is a fact about the town. Bayes turns those into the number you actually want.

Three classic traps

The taxi and the witness

A cab hit someone at night. 85% of the town’s cabs are green, 15% are blue. A witness says the cab was blue, and tests show he identifies colours correctly 80% of the time. Most people answer “80% likely blue”. Bayes says:

Of every 100 cabs, 12 are blue and correctly called blue, but 17 are green and wrongly called blue. The rare base rate wins.

Steve the librarian

Steve is shy, tidy and loves detail. Librarian or farmer? The description fits librarians better: say 40% of librarians but only 10% of farmers. But there are about 20 farmers for every librarian, so out of 21 people the expected matches are $1 \times 0.4 = 0.4$ librarians against $20 \times 0.1 = 2$ farmers. Steve is a farmer with probability about $2 / 2.4 \approx 83\%$.

A 99% accurate test

1% of people have a disease. A test catches 99% of sick people and falsely flags 1% of healthy ones. If you test positive, the chance you are sick is

A coin flip, despite “99% accuracy”. A second independent positive changes things fast: with $k$ positives the likelihoods are raised to the power $k$, and two positives already give 99%. Set the repeats slider in the figure to see it.

The fast way: odds times likelihood ratio

Bayes is cleaner in odds. Divide the formula for $H$ by the one for $\neg H$ and the evidence term cancels:

Posterior odds are prior odds times the likelihood ratio. For the coins: prior odds $1:999$, likelihood ratio $1024:1$, posterior odds $1024:999$. Independent pieces of evidence just multiply their likelihood ratios together, which is how you do this in your head.

In code

def posterior(prior, p_e_given_h, p_e_given_not_h, repeats=1):
  a = prior * p_e_given_h ** repeats
  b = (1 - prior) * p_e_given_not_h ** repeats
  return a / (a + b)

print(posterior(0.15, 0.8, 0.2))           # taxi: 0.414
print(posterior(0.01, 0.99, 0.01))         # one positive test: 0.5
print(posterior(0.01, 0.99, 0.01, 2))      # two positives: 0.99
print(posterior(1/1000, 1.0, 0.5, 10))     # 1000 coins: 0.506

# Check the taxi answer by simulation
import numpy as np
rng = np.random.default_rng(0)
blue = rng.random(1_000_000) < 0.15
says_blue = np.where(blue, rng.random(blue.size) < 0.8, rng.random(blue.size) < 0.2)
print(blue[says_blue].mean())              # ≈ 0.414