Download this Paper Open PDF in Browser

Generalized Stochastic Gradient Learning

40 Pages Posted: 21 Nov 2005  

George W. Evans

University of Oregon - Department of Economics; University of St. Andrews - School of Economics and Finance

Seppo Honkapohja

Bank of Finland; Centre for Economic Policy Research (CEPR); CESifo (Center for Economic Studies and Ifo Institute for Economic Research)

Noah Williams

Princeton University - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 3 versions of this paper

Date Written: October 2005

Abstract

We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.

Keywords: adaptive learning, E-stability, recursive least squares, robust estimation

JEL Classification: C62, C65, D83, E10, E17

Suggested Citation

Evans, George W. and Honkapohja, Seppo and Williams, Noah, Generalized Stochastic Gradient Learning (October 2005). CESifo Working Paper No. 1576. Available at SSRN: https://ssrn.com/abstract=852504

George W. Evans

University of Oregon - Department of Economics ( email )

1285 University of Oregon
Eugene, OR 97403
United States
541-346-4662 (Phone)
541-346-1243 (Fax)

University of St. Andrews - School of Economics and Finance ( email )

The Scores, Castlecliff
St. Andrews, Fife KY16 8RD
United Kingdom
44-1334-462435 (Phone)

HOME PAGE: http://www.st-andrews.ac.uk/economics/staff/pages.g.evans.shtml

Seppo Honkapohja (Contact Author)

Bank of Finland ( email )

P.O. Box 160
FIN-00101 Helsinki
Finland

Centre for Economic Policy Research (CEPR)

77 Bastwick Street
London, EC1V 3PZ
United Kingdom

CESifo (Center for Economic Studies and Ifo Institute for Economic Research)

Poschinger Str. 5
Munich, DE-81679
Germany

HOME PAGE: http://www.CESifo.de

Noah Williams

Princeton University - Department of Economics ( email )

Fisher Hall
Princeton, NJ 08544-1021
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Paper statistics

Downloads
63
Rank
253,485
Abstract Views
843