Towards Explaining Deep Learning: Asymptotic Properties of ReLU FFN Sieve Estimators

62 Pages Posted: 27 Dec 2019 Last revised: 6 Sep 2022

See all articles by Frank J. Fabozzi

Frank J. Fabozzi

Johns Hopkins University

Hasan Fallahgoul

Monash University ; Monash University

Vincentius Franstianto

Monash University

Gregoire Loeper

BNP Paribas; Monash University - School of Mathematical Sciences; Monash University - Monash Centre for Quantitative Finance and Investment Strategies; Ecole Polytechnique, Palaiseau - CMAP CNRS-UMR 7641 and Ecole Polytechnique

Date Written: December 6, 2019

Abstract

Recently, machine learning algorithms have increasing become popular tools for economic and financial forecasting. While there are several machine learning algo- rithms for doing so, a powerful and efficient algorithm for forecasting purposes is the multi-layer, multi-node neural network with rectified linear unit (ReLU) activa- tion function – deep neural network (DNN). Studies have demonstrated the empir- ical applications of DNN but have devoted less research to investigate its statistical properties which is mainly due to its severe nonlinearity and heavy parametrization. By borrowing tools from a non-parametric regression framework, sieve estimator, we first show that there exists such a sieve estimator for a DNN. We next establish three asymptotic properties of the ReLU network: consistency, sieve-based convergence rate, and asymptotic normality, and then validate our theoretical results using Monte Carlo analysis.

Keywords: Deep Learning, Neural Networks, Rectified Linear Unit, Sieve Estimators, Consistency, Rate of Convergence

JEL Classification: C1, C5

Suggested Citation

Fabozzi, Frank J. and Fallahgoul, Hasan A and Fallahgoul, Hasan A and Franstianto, Vincentius and Loeper, Gregoire, Towards Explaining Deep Learning: Asymptotic Properties of ReLU FFN Sieve Estimators (December 6, 2019). Available at SSRN: https://ssrn.com/abstract=3499324 or http://dx.doi.org/10.2139/ssrn.3499324

Frank J. Fabozzi

Johns Hopkins University ( email )

Baltimore, MD 20036-1984
United States

Hasan A Fallahgoul (Contact Author)

Monash University ( email )

Clayton Campus
Victoria, 3800
Australia

HOME PAGE: http://www.hfallahgoul.com

Monash University ( email )

Clayton Campus
Victoria, 3800
Australia

HOME PAGE: http://www.hfallahgoul.com

Vincentius Franstianto

Monash University ( email )

23 Innovation Walk
Wellington Road
Clayton, Victoria 3800
Australia

Gregoire Loeper

BNP Paribas ( email )

Paris
France

Monash University - School of Mathematical Sciences ( email )

Clayton Campus
Victoria, 3800
Australia

Monash University - Monash Centre for Quantitative Finance and Investment Strategies ( email )

9 Rainforest Walk
Clayton Campus
Monash University, Victoria 3800
Australia

HOME PAGE: http://https://www.monash.edu/science/quantitative-finance

Ecole Polytechnique, Palaiseau - CMAP CNRS-UMR 7641 and Ecole Polytechnique ( email )

Route de Saclay
Palaiseau, 91128
France

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
332
Abstract Views
2,527
Rank
166,135
PlumX Metrics