Backtesting Strategies Based on Multiple Signals

43 Pages Posted: 13 Jul 2015 Last revised: 12 Oct 2024

See all articles by Robert Novy-Marx

Robert Novy-Marx

Simon Business School, University of Rochester; National Bureau of Economic Research (NBER)

Date Written: July 2015

Abstract

Strategies selected by combining multiple signals suffer severe overfitting biases, because underlying signals are typically signed such that each predicts positive in-sample returns. “Highly significant” backtested performance is easy to generate by selecting stocks on the basis of combinations of randomly generated signals, which by construction have no true power. This paper analyzes t-statistic distributions for multi-signal strategies, both empirically and theoretically, to determine appropriate critical values, which can be several times standard levels. Overfitting bias also severely exacerbates the multiple testing bias that arises when investigators consider more results than they present. Combining the best k out of n candidate signals yields a bias almost as large as those obtained by selecting the single best of nk candidate signals.

Suggested Citation

Novy-Marx, Robert, Backtesting Strategies Based on Multiple Signals (July 2015). NBER Working Paper No. w21329, Available at SSRN: https://ssrn.com/abstract=2629935

Robert Novy-Marx (Contact Author)

Simon Business School, University of Rochester ( email )

Rochester, NY 14627
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
533
Abstract Views
2,720
Rank
103,755
PlumX Metrics