Survivorship Bias Means You Are Only Studying the Ones That Made It
Datasets quietly delete their failures. Any conclusion drawn from what remains describes a population that was selected for success.
The Mechanism
Survivorship bias occurs when a sample includes only entities that made it to the present, and the ones that did not are absent rather than recorded as failures.
The exclusion is rarely deliberate. It is what happens naturally when a database contains currently existing things. Funds that closed are no longer listed. Companies that went bankrupt are no longer in the index. Strategies that stopped working are no longer being marketed.
Every average computed from that sample is an average of survivors.
The Wartime Example
The clearest illustration comes from a statistical problem during the Second World War. Analysts examined returning aircraft, mapped where the bullet holes clustered, and proposed reinforcing those areas.
The statistician Abraham Wald pointed out the error. The holes marked places an aircraft could be hit and still return. The areas with no holes on surviving planes were precisely the areas where damage was fatal, and those were the areas to armour.
The missing observations carried the information. That is the general shape of the problem, and the missing observations are always the hard ones to notice.
Where It Distorts Finance
| Setting | What disappears | Effect |
|---|---|---|
| Fund databases | Closed and liquidated funds | Category returns overstated |
| Index history | Delisted and bankrupt members | Long run returns overstated |
| Backtests | Companies no longer listed | Strategy performance inflated |
| Manager track records | Discontinued strategies | Skill overstated |
Fund databases are the most studied case. Funds close primarily because they performed badly, so removing them removes the worst results. Studies estimating the size of this effect have found it adds a meaningful amount to reported category returns, enough to change conclusions about whether a category outperformed.
The Incubation Variant
A related practice makes it worse. A firm launches several small funds quietly, runs them for a period, then markets the ones that performed well and closes the rest.
The surviving fund has a genuine track record. It is also the outcome of a selection process the investor cannot see, and its history is the result of choosing winners after the fact rather than of skill applied in advance.
Backtesting Is Where It Bites
Testing a strategy on historical data requires a universe of securities to test on. If that universe is built from companies that exist today, every company that failed during the test period is missing.
A strategy that would have bought companies which subsequently went bankrupt never gets the chance to buy them, because they are not in the data. The backtest reports returns that could not have been earned by anyone operating at the time.
Proper databases include delisted securities with the reason and date. Using one that does not is a common and serious error, and it flatters value oriented and small company strategies most, since those categories contain the highest failure rates.
Beyond Finance
The pattern generalises. Studies of successful companies looking for common traits examine only successes, so any trait shared by both successes and failures appears to be a cause of success.
The same applies to advice from successful founders, investors, and executives. The sample is drawn entirely from people whose approach worked, and the identical approach in people it did not work for produced no book.
How to Check
Ask what happened to the entities that are not in the data. Ask whether the database includes dead securities and closed funds. Ask when the sample was selected relative to the period being studied. And be suspicious of any long run return figure where the constituent list was drawn at the end of the period rather than at the beginning.
The Bottom Line
Survivorship bias makes any sample of survivors look better than the population it came from, because the failures were removed rather than recorded. It inflates fund category returns, index histories, backtests, and every conclusion drawn from studying successful companies. The correction is always the same: find out what is missing and whether it was removed for reasons connected to performance.