Generalized Stochastic Gradient Learning

47 Pages Posted: 19 Sep 2007 Last revised: 16 May 2021

See all articles by George W. Evans

George W. Evans

University of Oregon - Department of Economics; University of St. Andrews - School of Economics and Finance

Seppo Honkapohja

Centre for Economic Policy Research (CEPR); Aalto University School of Business; CESifo (Center for Economic Studies and Ifo Institute for Economic Research); Aalto University - School of Business

Noah Williams

Princeton University - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 3 versions of this paper

Date Written: October 2005

Abstract

We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.

Suggested Citation

Evans, George W. and Honkapohja, Seppo and Honkapohja, Seppo and Williams, Noah, Generalized Stochastic Gradient Learning (October 2005). NBER Working Paper No. t0317, Available at SSRN: https://ssrn.com/abstract=827967

George W. Evans (Contact Author)

University of Oregon - Department of Economics ( email )

1285 University of Oregon
Eugene, OR 97403
United States
541-346-4662 (Phone)
541-346-1243 (Fax)

University of St. Andrews - School of Economics and Finance ( email )

The Scores, Castlecliff
St. Andrews, Fife KY16 8RD
United Kingdom
44-1334-462435 (Phone)

HOME PAGE: http://www.st-andrews.ac.uk/economics/staff/pages.g.evans.shtml

Seppo Honkapohja

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Aalto University School of Business ( email )

, PO Box 21210
Aalto FI-00076
Finland

CESifo (Center for Economic Studies and Ifo Institute for Economic Research)

Poschinger Str. 5
Munich, DE-81679
Germany

HOME PAGE: http://www.CESifo.de

Aalto University - School of Business

Finland

Noah Williams

Princeton University - Department of Economics ( email )

Fisher Hall
Princeton, NJ 08544-1021
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
28
Abstract Views
850
PlumX Metrics