Find the leaking bar
Unit 2 told you not to peek. Now find out whether you did, because look-ahead does not announce itself — it announces a great strategy.
The symptom is success
A leaked backtest looks like the best thing you have ever built. Smooth curve, shallow drawdowns, a hit rate in the seventies. That is the tell. If a first attempt at a simple rule produces a result that would make you rich, the prior should be a bug rather than an edge, and the next hour goes into looking for it.
Real edges are small, noisy, and mostly gone after costs. A result that does not look like that is a claim about your code, not about the market.
Four tests, and you write all of them
This is the course's craft discipline at full strength. Bias detection is code review, and code review that runs automatically is a test.
1. The future-mutation test. Compute all signals. Change every bar after index k to nonsense. Recompute. Every signal at or before k must be bit-identical. This is the general form of the test from unit 2 and it catches leaks no eyeball review will.
2. The truncation test. Run the backtest on bars 0..k, then on all bars, and compare the signal and position series through k — not the trade lists, because the truncated run force-closes an open position at k and the full run does not, which is a difference by construction rather than a bug. If running with more data changes what happened in the past, something is reading ahead.
3. The fill-range assertion. From the previous lesson: every fill inside its bar's low-to-high.
Those three are leak detectors. The fourth is a different instrument and it is worth keeping them separate in your head.
4. The shuffle test — a null test, not a leak test. Replace the returns with a random permutation of themselves, keeping the same dates, and run. A rule that depends on chronological structure should lose most of its edge. If it performs just as well on shuffled data, the "edge" comes from exposure, the return distribution or your accounting rather than from anything about the order of the series.
It does not detect look-ahead: a rule that reads tomorrow's return will happily read tomorrow's shuffled return and score just as well. Use it to answer "is there any time structure here at all", and use the first three to answer "am I cheating".
Where the leaks actually come from
In order of how often each one appears in generated code:
- A window that extends past the current bar. A window ending at bar
iis honest for a signal you act on ati + 1; anything reachingi + 1or beyond is reading the future. One index. - Normalising over the whole series — dividing by the full-sample mean, or scaling to the maximum. Every point then knows the future.
- A parameter chosen by looking at the whole sample, which is the human version of the same leak. This is unit 4's subject.
fillnaor interpolation running backwards, quietly filling today's hole with tomorrow's value.- Sorting or reindexing that reverses the series without anyone noticing.
Ask your assistant to attack it
Hand it your strategy function and the four tests and ask: "find a way this could be using information from the future." Then read the answers sceptically — some will be wrong, and one or two will be things you would not have found.
This is the most valuable single prompt in the course. Assistants are good at this specific task because it is pattern-matching over a well-known bug family.
Try it now
Write all four tests and run them. Then introduce each of the five leaks above on purpose, one at a time, and confirm which test catches it. You will find at least one leak that none of them catch — write the fifth test for that one, and you have learned more about testing than any tutorial teaches.