Skip to content
GetProfitable
Search
Dictionary

Significance level

The p-value threshold you commit to before testing, conventionally 0.05, below which you are willing to reject the null hypothesis.

The 5% convention is arbitrary and borrowed from agricultural statistics in the 1920s. It means you accept being wrong one time in twenty when the null is actually true, which is a fine trade when a failed experiment costs a field of barley and a poor one when it costs a live account.

In strategy research the threshold should be much tighter because you are testing many ideas against noisy data. Practitioners who take this seriously use 0.01 or apply a bonferroni-correction, and treat anything between 0.01 and 0.05 as a reason to gather more data rather than to deploy.

Note the asymmetry with type-ii-error: tightening the level reduces false discoveries but increases the chance of discarding a real but modest edge. With trading you can usually live with that, because a strategy too weak to clear a strict test is also too weak to survive slippage.

Related: p-value, type-i-error, type-ii-error, bonferroni-correction

Educational only, not advice. Spotted an error? Post in Site Feedback.