A backtest asks how a rule would have performed on historical data. The result is useful only if the rule, information, and trading assumptions could have existed at the time. Otherwise the test answers a different question: how well can we explain the past after seeing it?
What was known at the time
Suppose a strategy buys companies with rising quarterly profits. A quarter ending March 31 does not mean its results were known on March 31. The test must wait until the relevant announcement or filing was available, then allow time to trade.
The same principle applies to revised economic statistics, index membership, delisted stocks, and cleaned data. Using today's surviving companies for a historical test quietly removes some failures. Using a revised data series can give a strategy information that contemporary traders never saw.
For each signal, ask for its original publication time, the earliest plausible order time, and the price available then. A model using the final daily close generally cannot also assume a fill at that exact close unless the signal and order process genuinely permitted it.
The search behind the winning result
Testing 20 thresholds, 10 holding periods, and five universes creates 1,000 combinations. The best result may look impressive even if none has a dependable advantage. Reporting only the winner hides the search that produced it.
Bailey and co-authors' research on backtest overfitting examines this selection problem. Their related Deflated Sharpe Ratio paper addresses the effect of multiple trials and non-normal returns on apparent performance.
You do not need to implement every statistical method to adopt the discipline. Keep a record of rejected variants. Reserve later data for a rule whose settings are already fixed. If you repeatedly inspect that reserved period and adjust the rule, it has become part of development rather than an independent test.
After execution costs
Imagine a signal earns an average 0.30% before costs. Estimated slippage is 0.18%, fees 0.04%, and financing 0.03%. The net is only 0.05%. Underestimating slippage by another 0.06 percentage points turns the result into a 0.01% loss.
That calculation suggests where the investigation belongs: executable liquidity and timing, rather than another decimal place in the signal. Use bid and ask prices where available, size-dependent execution assumptions, and realistic borrow or funding. A trade that requires shorting unavailable shares is not implementable merely because a price series exists.
Capacity also matters. A rule that could trade $1,000 without moving a market may not behave similarly at $1 million. Historical volume is useful context, but it does not guarantee that liquidity was available at the exact time and direction required.
Return, exposure, and robustness
Compare with a benchmark that carries similar exposure. A strategy with twice the market sensitivity should not receive full credit for earning more during a bull market. Examine drawdowns, time underwater, turnover, concentration, and the largest individual contributors.
Then vary assumptions modestly. Does a nearby threshold produce a similar result, or does performance collapse unless the setting is exactly 17? A broad region of acceptable results is easier to reconcile with an economic mechanism than one isolated optimum. It is supporting evidence, not a guarantee.
Do not confuse trade count with independent observations. Several positions opened after the same macro announcement may be versions of one bet. Consecutive returns can also be related, making uncertainty larger than a calculation that assumes independent outcomes would suggest.
Why the return might persist
A credible test should explain who is providing the return and why the opportunity might remain available: compensation for risk, a constraint on other investors, or a repeatable information advantage. It should also name the conditions under which that explanation would stop working.
The next step is a prospective record with the rule frozen and assumptions visible. The most valuable backtest is one that makes a future disappointment diagnosable. If the only explanation for its success is that this combination had the best historical chart, there is little basis for expecting the chart to continue.