40 Pages Posted: 21 Nov 2005
Date Written: October 2005
We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.
Keywords: adaptive learning, E-stability, recursive least squares, robust estimation
JEL Classification: C62, C65, D83, E10, E17
Suggested Citation: Suggested Citation
Evans, George W. and Honkapohja, Seppo and Williams, Noah, Generalized Stochastic Gradient Learning (October 2005). CESifo Working Paper No. 1576. Available at SSRN: https://ssrn.com/abstract=852504