Text Selection

57 Pages Posted: 10 Dec 2019

See all articles by Bryan T. Kelly

Bryan T. Kelly

Yale SOM; AQR Capital Management, LLC; National Bureau of Economic Research (NBER)

Asaf Manela

Washington University in St. Louis - John M. Olin Business School; Reichman University

Alan Moreira

University of Rochester - Simon Business School

Multiple version iconThere are 2 versions of this paper

Date Written: November 22, 2019


Text data is ultra-high dimensional, which makes machine learning techniques indispensable for textual analysis. Text is often selected—journalists, speechwriters, and others craft messages to target their audiences’ limited attention. We develop an economically motivated high dimensional selection model that improves learning from text (and from sparse counts data more generally). Our model is especially useful when the choice to include a phrase is more interesting than the choice of how frequently to repeat it. It allows for parallel estimation, making it computationally scalable. A first application revisits the partisanship of US congressional speech. We find that earlier spikes in partisanship manifested in increased repetition of different phrases, whereas the upward trend starting in the 1990s is due to entirely distinct phrase selection. Additional applications show how our model can backcast, nowcast, and forecast macroeconomic indicators using newspaper text, and that it substantially improves out-of-sample fit relative to alternative approaches.

Keywords: Text analysis, machine learning, selection model, high dimension forecast, multinomial regression, hurdle, zero inflation, partisanship, intermediary capital

Suggested Citation

Kelly, Bryan T. and Manela, Asaf and Moreira, Alan, Text Selection (November 22, 2019). Available at SSRN: https://ssrn.com/abstract=3491942 or http://dx.doi.org/10.2139/ssrn.3491942

Bryan T. Kelly

Yale SOM ( email )

135 Prospect Street
P.O. Box 208200
New Haven, CT 06520-8200
United States

AQR Capital Management, LLC ( email )

Greenwich, CT
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Asaf Manela (Contact Author)

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States
314-935-9178 (Phone)

HOME PAGE: http://apps.olin.wustl.edu/faculty/manela

Reichman University ( email )


Alan Moreira

University of Rochester - Simon Business School ( email )

Rochester, NY 14627
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics