The Claude Quant PlaybookDownload ebookDownload ebook

p-value

intermediateStatistics7 of 10

The chance of a result at least as extreme as yours if there is really nothing there. Small p-values mean luck alone rarely produces such a result.

Build a trading system with Claude.Download ebook

01The formula

the p-value
the test statistic if the null hypothesis is true
the test statistic you observed
the null hypothesis, such as 'no edge'
the standard normal CDF
the number of independent strategies tried

02See it move

03Where it is used

04Why traders use it

It is the most quoted and most misread number in research. Understanding it, and how it degrades when you try many ideas, is the main defence against backtest overfitting.

05Worked example

A test statistic of 2.10 counting 2 tails gives p = 3.57% (one-sided 1.79%). But if 20 no-edge strategies were tried, the chance that at least one looks this good is 51.7%.

06How it is built

  1. Assume is true, so the statistic follows a known distribution, here the standard normal.
  2. Find the area beyond your observed value: one tail, or both tails for a two-sided test.
  3. That area is the p-value: the share of no-effect worlds that look at least this extreme.
  4. With independent tries, the chance that at least one reaches is .

07The common mistake

Reading p as the chance the null is true. A p-value of 3% does not mean a 97% chance the strategy works; it is computed assuming it does not.

08In Python and LaTeX

from scipy.stats import norm
z, m = 2.1, 20
p = 2 * norm.sf(z)
print(round(p, 4), round(1 - (1 - p)**m, 3))
p = P\big(|Z| \ge |z_{\text{obs}}| \mid H_0\big) = 2\,\big(1 - N(|z_{\text{obs}}|)\big), \qquad P(\text{at least one of } m) = 1 - (1 - p)^m

09Where it fits

Rather not code the maths yourself? Describe your trading idea and let AI build the bot.

Build a bot with AgenKit

The worked example and the model run the same code, so every number matches. Models simplify markets on purpose. This is education, not financial advice.