Sample size and Monte Carlo reshuffling
Lesson 18 · about 12 min
Thirty trades with a +0.4R expectancy is a coin-flip. Three hundred trades with the same expectancy is evidence. The difference is sample size, and there is a simple calculation that tells you which one you have. Once you know the expectancy is probably real, a second technique, Monte Carlo reshuffling, tells you how bad the drawdowns could plausibly get with that same edge.
Standard error of expectancy
The expectancy you measured is the average of a noisy sample. Its uncertainty is the standard error:
standard error = standard deviation of R ÷ √(number of trades)
A rough rule: the measured expectancy is convincingly different from zero when it is at least twice the standard error.
| Trades | Std dev of R | Standard error | Expectancy needed for 2 × SE |
|---|---|---|---|
| 30 | 1.8 | 0.33 | +0.66R |
| 50 | 1.8 | 0.25 | +0.51R |
| 100 | 1.8 | 0.18 | +0.36R |
| 200 | 1.8 | 0.13 | +0.25R |
| 400 | 1.8 | 0.09 | +0.18R |
| 1,000 | 1.8 | 0.06 | +0.11R |
A strategy with a genuine +0.25R expectancy and a typical 1.8R standard deviation needs about 200 trades before the backtest can distinguish it from luck. A strategy with +0.1R needs over a thousand. This is why very short-term systems with small per-trade edges require enormous samples, and why a swing system showing +0.4R on 35 trades is not yet showing anything.
The same logic applies to win rate. The standard error of a proportion is √(p × (1 − p) ÷ n). At n = 50 and p = 0.5, that is 0.07, so a measured 55% win rate is within one standard error of 48%.
What sample size cannot fix
More trades from the same period do not help if the period is one regime. Two thousand 1-minute trades from a single month all sampled the same volatility, the same trend, the same participants. Sample size counts independent observations, and trades close together in time are not independent. For intraday systems, count the number of distinct days or weeks as well as the number of trades, and treat the smaller one as your real sample.
Key idea: Standard error = std dev of R ÷ √n. Until the expectancy is at least twice the standard error, the backtest has not distinguished the strategy from luck. Most systems need 200 or more trades across more than one regime.
Monte Carlo reshuffling
Suppose the expectancy is real. The equity curve you saw is still only one ordering of those trades. Had the same trades arrived in a different order, the maximum drawdown would have been different, and possibly much worse. Reshuffling asks: given these trades, what range of drawdowns is plausible?
The procedure:
- Take the list of trade results in R.
- Shuffle the order randomly.
- Build the equity curve and record the maximum drawdown.
- Repeat 1,000 or more times.
- Look at the distribution of maximum drawdowns.
A variant, resampling with replacement, draws n trades from the list at random with repeats allowed. This also varies which trades appear, giving a slightly wider distribution, and is the better choice when a few outliers dominate.
A worked reshuffle
The original backtest: 180 trades, expectancy +0.22R, standard deviation 1.6R, observed max drawdown 9.8R.
| Percentile of simulated max drawdown | Max DD (R) |
|---|---|
| 5th (mild) | 7.1 |
| 25th | 9.4 |
| 50th (median) | 11.3 |
| 75th | 13.8 |
| 95th (severe) | 18.2 |
| 99th | 22.5 |
The observed 9.8R was on the lucky side of the median. With the same trades in a different order, a 14R drawdown is a one-in-four event and 18R is one in twenty. Module 8 uses the 95th percentile to set position size; for now, the point is that the backtest's own drawdown number is the least conservative estimate available.
The same simulation gives a distribution of final equity, which answers a different question: how likely is a losing year?
| Simulated 12-month outcome | Probability |
|---|---|
| Net loss | 14% |
| Between 0 and +10R | 22% |
| Between +10R and +25R | 41% |
| Above +25R | 23% |
A system with a real +0.22R edge over 60 trades a year still posts a losing year about one time in seven. Knowing this in advance is what stops you abandoning a working system after a bad year, and what stops you doubling size after a good one.
What reshuffling assumes
Reshuffling treats trades as independent and drawn from the same distribution. Neither is fully true: losses cluster in bad regimes, and the distribution shifts as markets change. The simulation therefore understates drawdowns somewhat. It is still far better than the single observed path. For a rougher but regime-aware check, reshuffle blocks of consecutive trades (say, ten at a time) rather than single trades.
Doing it in a spreadsheet
Put the trade results in a column. In a second column, generate random numbers; sort both columns by the random column to get a shuffled order; compute the running equity and its drawdown; record the max. A macro or a data table can repeat it. Python does it in a dozen lines. Module 7 gives the details.
Try it: Take your trade list, compute std dev of R ÷ √n, and compare with your expectancy. Then reshuffle the list 200 times (by hand with a random column and sorting if necessary) and record the maximum drawdown each time. Write down the 95th percentile. Compare it with the backtest's reported max drawdown.
Recap
- Standard error of expectancy is std dev of R ÷ √n; you want expectancy at least twice that.
- A +0.25R system with 1.8R std dev needs about 200 trades to be distinguished from luck.
- Trades close in time are not independent; count regimes and days as well as trades.
- Monte Carlo reshuffling shows the range of drawdowns the same trades could have produced.
- Use the 95th percentile drawdown, not the observed one, when planning size and survival.
See it drawn
Original diagrams for the ideas on this page. Illustrative, not real market data.