p-value
The chance of a result at least as extreme as yours if there is really nothing there. Small p-values mean luck alone rarely produces such a result.
01The formula
- the p-value
- the test statistic if the null hypothesis is true
- the test statistic you observed
- the null hypothesis, such as 'no edge'
- the standard normal CDF
- the number of independent strategies tried
02See it move
03Where it is used
- Quant researcherScreens signals by p-value, adjusting the bar for how many signals were tried.
- Risk managerUses p-values in model validation, for example to test value-at-risk breach counts.
- Portfolio managerReads research claims critically by asking what the p-value really means.
04Why traders use it
It is the most quoted and most misread number in research. Understanding it, and how it degrades when you try many ideas, is the main defence against backtest overfitting.
05Worked example
A test statistic of 2.10 counting 2 tails gives p = 3.57% (one-sided 1.79%). But if 20 no-edge strategies were tried, the chance that at least one looks this good is 51.7%.
06How it is built
- Assume is true, so the statistic follows a known distribution, here the standard normal.
- Find the area beyond your observed value: one tail, or both tails for a two-sided test.
- That area is the p-value: the share of no-effect worlds that look at least this extreme.
- With independent tries, the chance that at least one reaches is .
07The common mistake
Reading p as the chance the null is true. A p-value of 3% does not mean a 97% chance the strategy works; it is computed assuming it does not.
08In Python and LaTeX
from scipy.stats import norm z, m = 2.1, 20 p = 2 * norm.sf(z) print(round(p, 4), round(1 - (1 - p)**m, 3))
p = P\big(|Z| \ge |z_{\text{obs}}| \mid H_0\big) = 2\,\big(1 - N(|z_{\text{obs}}|)\big), \qquad P(\text{at least one of } m) = 1 - (1 - p)^m09Where it fits
Rather not code the maths yourself? Describe your trading idea and let AI build the bot.
Build a bot with AgenKitThe worked example and the model run the same code, so every number matches. Models simplify markets on purpose. This is education, not financial advice.