Deep Nothing: How to Hit the Wall With Deep Learning

Abstract

We present a worked out example in R (including sources in the appendix), where deep learning falls behind much simpler methods. It is an already published application of a LeNet style convolutional neural network (CNN) for image recognition. We show that this complex CNN is outperformed by a single layer perceptron and that a logistic regression comes close if done naively and also outperforms if a transformation is applied on the inputs. The reason for this is highlighted by visual data analysis.

Keywords: Deep Learning, Image Recognition, CNN, MLP, MXNet

JEL Classification: C01, C02, C19, C55