A
Fewer than 20 trades
Too few trades to draw a reliable conclusion — a handful of results can easily be dominated by chance.
Manual Backtesting and Historical Validation
Testing a rule against history one trade at a time is how a vague idea becomes a checked one. But the same process can quietly mislead if the sample is too small or the rule was tuned to fit the very data it's tested against.
- Explain why testing a rule trade-by-trade against historical charts is more informative than testing it in the abstract.
- Recognize why a larger sample of historical trades carries more weight than a small one.
- Identify curve-fitting: tuning a rule repeatedly to match the same dataset it's tested on.
- Distinguish in-sample testing from out-of-sample testing.
- Avoid treating a small winning sample as proof a rule works.
You chose parameters after seeing the history. What is the backtest still missing?
Backtesting applies a fixed rule to historical data. It is useful for finding flaws and estimating behavior, but it becomes misleading when parameters are repeatedly tuned to the same sample.
Manual Backtest
Stop cherry-picking charts: run all 24 historical samples
The rule is fixed. Each click reveals one more R outcome and updates the distribution metrics.
????????????????????????
Samples tested0/24
Win rate0.0%
Average win+0.00R
Average loss-0.00R
Max drawdown0.00R
Expectancy+0.00R
Checking a Rule Trade-by-Trade Turns an Idea Into Evidence
A rule-based entry and exit only becomes useful once it's actually checked against how it would have performed historically. Manually scrolling through historical charts and marking each time the rule would have triggered — and what happened afterward — surfaces real edge cases that thinking about a rule in the abstract tends to miss.
See How Confidence Should Scale With the Number of Trades Tested
Move the slider and read how a testing sample of that size is typically regarded — more trades reduce the chance a result is pure coincidence, but no sample size promises future results.
B
20 to 99 trades
An early read worth continued testing — the pattern is showing up more than a few times, but the sample is still fairly small.
C
100 trades or more
A larger sample that carries more weight, though even a large historical sample never guarantees the same results going forward.
Tuning a Rule to Fit the Same Data It's Tested On Risks Matching Noise
Adjusting a rule's exact thresholds repeatedly until it produces the best result on one specific stretch of historical data tends to capture coincidental patterns in that data — patterns that may never repeat. The fix is testing on data the rule never saw during design, often called out-of-sample testing.
Does This Testing Method Actually Support Its Conclusion?
Pick a case and judge whether the testing approach described is a meaningful check or a way to accidentally fool the tester.
A
Curve-fitting
A trader adjusts a rule's exact thresholds repeatedly until it produces the best possible result on the same historical data it was tested against. This risks curve-fitting — tuning the rule to match noise in one specific dataset rather than a pattern that holds more broadly.
B
A tiny sample
A trader tests a rule on 8 historical trades, all of which were profitable, and concludes the rule is proven. Eight trades is too small a sample to rule out chance, regardless of how many of them were profitable.
C
Out-of-sample testing
A trader designs a rule using one stretch of historical data, then tests it separately on a later stretch of data it was never shown during design. Testing on data the rule never saw during design is a more meaningful check than testing on the same data used to build it.
Four Checks for a Backtest You Can Interpret
1
Sample size
How many historical trades were actually tested?
2
In-sample vs. out-of-sample
Was any of the testing done on data the rule never saw during design?
3
Threshold stability
Were exact numbers tuned repeatedly to fit this same dataset?
4
Rule consistency
Was the rule applied the same way on every historical trade, without exceptions?
Make the Backtest Answer a Narrow, Reproducible Question
A handful of trades can look convincing purely by chance.
Out-of-sample testing checks whether a rule generalizes.
A rule perfectly tuned to one dataset may just describe that dataset's coincidences.
Put Your Understanding to the Test
Submit your answers to see detailed explanations.
Question 1 of 3
A trader tests a rule on 8 historical trades, all profitable, and concludes it's proven. What is the issue?
Question 2 of 3
A trader repeatedly adjusts a rule's exact thresholds until it produces the best possible result on the same historical data it's tested against. What risk does this create?
Question 3 of 3
A rule's thresholds were tuned on three years of data. Run unchanged on a later stretch it never saw, the result is clearly weaker but still positive. How should the two runs be read together?
Stuck? Ask Mira to Break It Down
Describe your rule and how many historical trades you've checked, and Mira can help you think through sample size and curve-fitting risk — it won't run a backtest on live market data for you.
Checking sign-in status...
Put this concept into practice
A Strategy Isn't 'I Think It'll Go Up'
Distinguish 'I have a feeling it'll rise' from 'a repeatable, reviewable, testable set of trading rules', and state that a complete strategy has to start with clear entry, risk, and exit conditions.
A Backtest Isn't a Crystal Ball
Understand that a backtest applies rules to historical data to examine past performance and risk characteristics, but it can't prove future profitability, and ignoring costs overstates performance.