Market Efficiency in the Age of Big Data

52 Pages Posted: 31 Dec 2019 Last revised: 3 Dec 2021

See all articles by Ian Martin

Ian Martin

London School of Economics & Political Science (LSE) - Department of Finance

Stefan Nagel

University of Chicago - Booth School of Business; National Bureau of Economic Research (NBER); Centre for Economic Policy Research; CESifo (Center for Economic Studies and Ifo Institute)

Multiple version iconThere are 3 versions of this paper

Date Written: December 2019

Abstract

Modern investors face a high-dimensional prediction problem: thousands of observable variables are potentially relevant for forecasting. We reassess the conventional wisdom on market efficiency in light of this fact. In our model economy, which resembles a typical machine learning setting, N assets have cash flows that are a linear function of J firm characteristics, but with uncertain coefficients. Risk-neutral Bayesian investors impose shrinkage (ridge regression) or sparsity (Lasso) when they estimate the J coefficients of the model and use them to price assets. When J is comparable in size to N, returns appear cross-sectionally predictable using firm characteristics to an econometrician who analyzes data from the economy ex post. A factor zoo emerges even without p-hacking and data-mining. Standard in-sample tests of market efficiency reject the no-predictability null with high probability, despite the fact that investors optimally use the information available to them in real time. In contrast, out-of-sample tests retain their economic meaning.

Suggested Citation

Martin, Ian W. R. and Nagel, Stefan, Market Efficiency in the Age of Big Data (December 2019). NBER Working Paper No. w26586, Available at SSRN: https://ssrn.com/abstract=3511296

Ian W. R. Martin (Contact Author)

London School of Economics & Political Science (LSE) - Department of Finance ( email )

United Kingdom

HOME PAGE: http://personal.lse.ac.uk/martiniw/

Stefan Nagel

University of Chicago - Booth School of Business ( email )

5807 S. Woodlawn Avenue
Chicago, IL 60637
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Centre for Economic Policy Research ( email )

London
United Kingdom

CESifo (Center for Economic Studies and Ifo Institute) ( email )

Poschinger Str. 5
Munich, DE-81679
Germany

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
66
Abstract Views
994
rank
191,520
PlumX Metrics