Which backtest errors invent a factor?
A factor is a backtest that survived. The quant-coding domain's backtesting course taught the leaking bar and survivorship on a single strategy; here they are the specific ways a factor gets invented, because each one manufactures a premium out of nothing, and each has been found in a published paper.
Survivorship: the companies that are not in your list
Build a value sort on today's listed companies and run it back twenty years. The cheap group is full of companies that were cheap in 2006 and are still listed in 2026 — the survivors. The ones that were cheap because they were failing, and then failed, are not in the list, so the sort never bought them. A value premium built that way is the premium of cheap companies that turned out fine, which is not a rule anyone can follow forward.
The cure is a universe that includes delistings, with the delisting return — often near total loss — charged to the portfolio. Shumway showed in 1997 that omitting delisting returns overstated the size premium materially, because the smallest companies delist most.
Look-ahead: the accounts that were not yet published
A value sort on 31 December uses book value at 31 December. But the accounts for a December year-end are published in February or March, and a sort built on them in January is using numbers nobody had. Fama and French's convention — sort in June on the previous December's accounts — exists to close that gap. A backtest that ignores it earns a premium from information arriving early.
Prices leak too. A momentum sort that ranks on the close and trades at the same close has executed at a price it could not have known; the backtesting course's leaking bar is this, and the fix — rank on today, trade tomorrow — costs a day of return that a paper factor keeps.
Data mining: the sort that was tried three hundred times
The zoo lesson's t-statistic of 3 is the guard here. A characteristic that was the fortieth one tested on the same data is not a discovery, and a factor whose definition was tuned — twelve months rather than eleven, the 30th percentile rather than the 25th — until the backtest looked best has been fitted, not found. The test is out-of-sample: other decades, other countries, definitions nobody tuned.
The honest checklist
Before believing a factor backtest, four questions. Does the universe include the companies that disappeared, with their delisting returns? Are the accounts used on the date they were public? Are the trades at prices available after the signal? And was this definition chosen before the test or after it? A published factor that passes all four is worth reading further; most of the three hundred fail at least one, and the failure is usually the reason they were published.
In the data
Every set of company accounts carries two dates: the period it covers and the day it was filed. Apple's newest annual balance sheet:
A sort can use accounts on the filing date, the day they were public, rather than the day they cover; that closes the look-ahead gap. The delisted names are the harder part: a survivorship-clean universe needs the companies that stopped trading and the tickers that changed hands, which is the list the backtesting course built.
Try it now
From Apple's two dates above, write down how many weeks apart they are. A sort on the period end date, for those weeks, used numbers nobody had.
Reopen Find the leaking bar and write down its rule for the signal date and the trade date. Then apply it to the momentum recipe from the previous unit — which of the three closes moves by a day?
The index a clean universe reconstitutes against, for scale: