Generalized Stochastic Gradient Learning

40 Pages Posted: 21 Nov 2005

See all articles by George W. Evans

George W. Evans

University of Oregon - Department of Economics; University of St. Andrews - School of Economics and Finance

Seppo Honkapohja

Centre for Economic Policy Research (CEPR); Aalto University School of Business; CESifo (Center for Economic Studies and Ifo Institute for Economic Research); Aalto University - School of Business

Noah Williams

Princeton University - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 3 versions of this paper

Date Written: October 2005

Abstract

We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.

Keywords: adaptive learning, E-stability, recursive least squares, robust estimation

JEL Classification: C62, C65, D83, E10, E17

Suggested Citation

Evans, George W. and Honkapohja, Seppo and Honkapohja, Seppo and Williams, Noah, Generalized Stochastic Gradient Learning (October 2005). CESifo Working Paper No. 1576, Available at SSRN: https://ssrn.com/abstract=852504 or http://dx.doi.org/10.2139/ssrn.852504

George W. Evans

University of Oregon - Department of Economics ( email )

1285 University of Oregon
Eugene, OR 97403
United States
541-346-4662 (Phone)
541-346-1243 (Fax)

University of St. Andrews - School of Economics and Finance ( email )

The Scores, Castlecliff
St. Andrews, Fife KY16 8RD
United Kingdom
44-1334-462435 (Phone)

HOME PAGE: http://www.st-andrews.ac.uk/economics/staff/pages.g.evans.shtml

Seppo Honkapohja (Contact Author)

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Aalto University School of Business ( email )

, PO Box 21210
Aalto FI-00076
Finland

CESifo (Center for Economic Studies and Ifo Institute for Economic Research)

Poschinger Str. 5
Munich, DE-81679
Germany

HOME PAGE: http://www.CESifo.de

Aalto University - School of Business

Finland

Noah Williams

Princeton University - Department of Economics ( email )

Fisher Hall
Princeton, NJ 08544-1021
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
71
Abstract Views
1,122
rank
354,555
PlumX Metrics