Backtesting
Read the paperopens papers.ssrn.com in a new tab
What they found
The more detailed companion to 'Evaluating Trading Strategies'. The authors formalize the multiple-testing framework for backtests, compare methods for controlling the family-wise error rate and the false discovery rate, and show how to apply them when the number of tried strategies is unknown by modeling the search process. They apply the framework to the published factor literature and to a set of trading strategies, showing how the required t-statistic rises with the number of tests and how many published results fall below it.
What you can use
- Even if you do not know exactly how many strategies you tried, you can model the search and still correct for it.
- False discovery rate control (accepting some false positives to keep more true ones) is usually more useful for traders than the strict family-wise correction.
- The framework applies to your own strategy development as much as to academic factors.
Caveats
Statistical assumptions about the distribution of true and false strategies drive the results. Aimed at quantitative readers. SSRN version linked.
Tags: backtesting, multiple-testing, false-discovery-rate, statistics
Summaries are our own reading of the paper, not the authors' words. Educational only, not advice. Discuss it in Book Club.