Deep Regression Ensembles

29 Pages Posted: 8 Mar 2022

See all articles by Antoine Didisheim

Antoine Didisheim

The University of Melbourne; Swiss Finance Institute

Bryan T. Kelly

Yale SOM; AQR Capital Management, LLC; National Bureau of Economic Research (NBER)

Semyon Malamud

Ecole Polytechnique Federale de Lausanne; Centre for Economic Policy Research (CEPR); Swiss Finance Institute

Date Written: March 3, 2022


We introduce a methodology for designing and training deep neural networks (DNN) that we call 'Deep Regression Ensembles' (DRE). It bridges the gap between DNN and two-layer neural networks trained with random feature regression. Each layer of DRE has two components, randomly drawn input weights and output weights trained myopically (as if the final output layer) using linear ridge regression. Within a layer, each neuron uses a different subset of inputs and a different ridge penalty, constituting an ensemble of random feature ridge regressions. Our experiments show that a single DRE architecture is at par with or exceeds state-of-the-art DNN in many data sets. Yet, because DRE neural weights are either known in closed-form or randomly drawn, its computational cost is orders of magnitude smaller than DNN.

Keywords: Deep learning, Neural network, Random features, Ensembles

Suggested Citation

Didisheim, Antoine and Kelly, Bryan T. and Malamud, Semyon, Deep Regression Ensembles (March 3, 2022). Swiss Finance Institute Research Paper No. 22-20, 2022, Available at SSRN: or

Antoine Didisheim

The University of Melbourne ( email )

Parkville, 3010
0435776821 (Phone)

Swiss Finance Institute ( email )

University of Melbourne
Melbourne, VA
0797605012 (Phone)

Bryan T. Kelly

Yale SOM ( email )

135 Prospect Street
P.O. Box 208200
New Haven, CT 06520-8200
United States

AQR Capital Management, LLC ( email )

Greenwich, CT
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Semyon Malamud (Contact Author)

Ecole Polytechnique Federale de Lausanne ( email )

Lausanne, 1015

Centre for Economic Policy Research (CEPR) ( email )

United Kingdom

Swiss Finance Institute

c/o University of Geneva
40, Bd du Pont-d'Arve
CH-1211 Geneva 4

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics