Skip to content
GetProfitable
Search
Dictionary

Parameter sensitivity

How much a strategy's results change when you nudge its inputs; a robust strategy degrades gracefully, a fitted one collapses.

The test is simple. Re-run the backtest across a grid of nearby parameter values and look at the whole surface instead of the peak. Report the median result of the neighbourhood, not the best cell.

Example: a lookback grid from 10 to 100 in steps of 5 produces annual returns of 3%, 5%, 4%, 6%, 31%, 5%, 2%. The 31% is a spike surrounded by noise and should be treated as luck. A surface reading 11%, 13%, 14%, 15%, 14%, 12% is far less exciting and far more likely to survive.

Sensitivity analysis costs nothing but compute and catches more bad strategies than any other single check, including out-of-sample testing, because it works even when your sample is short.

Related: parameter, parameter-plateau, overfitting, robustness

Educational only, not advice. Spotted an error? Post in Site Feedback.