Generalized Stochastic Gradient Learning

Evans, George W.; Honkapohja, Seppo; Williams, Noah

Download This Paper

Open PDF in Browser

Add Paper to My Library

Generalized Stochastic Gradient Learning

NBER Working Paper No. t0317

47 Pages Posted: 19 Sep 2007 Last revised: 16 May 2021

See all articles by George W. Evans

George W. Evans

University of Oregon - Department of Economics; University of St. Andrews - School of Economics and Finance

Seppo Honkapohja

Centre for Economic Policy Research (CEPR); Aalto University School of Business; CESifo (Center for Economic Studies and Ifo Institute for Economic Research); Aalto University - School of Business

Noah Williams

Princeton University - Department of Economics; National Bureau of Economic Research (NBER)

There are 3 versions of this paper

Date Written: October 2005

Abstract

We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.

Suggested Citation: Suggested Citation

Evans, George W. and Honkapohja, Seppo and Honkapohja, Seppo and Williams, Noah, Generalized Stochastic Gradient Learning (October 2005). NBER Working Paper No. t0317, Available at SSRN: https://ssrn.com/abstract=827967