Gradient Estimation Schemes for Noisy Functions

CentER Working Paper No. 2003-12

Posted: 26 May 2004

See all articles by R.C.M. Brekelmans

R.C.M. Brekelmans

Tilburg University - Tilburg University School of Economics and Management

Lonneke Driessen

Centre for Quantitative Methods

Herbert Hamers

Tilburg University - Center for Economic Research (CentER); Tilburg University - Department of Econometrics & Operations Research

Dick den Hertog

Tilburg University - Department of Econometrics & Operations Research

Date Written: February 2003

Abstract

In this paper we analyze different schemes for obtaining gradient estimates when the underlying function is noisy. Good gradient estimation is e.g. important for nonlinear programming solvers. As an error criterion we take the norm of the difference between the real and estimated gradients. This error can be split up into a deterministic and a stochastic error. For three finite difference schemes and two Design of Experiments (DoE) schemes we analyze both the deterministic and the stochastic errors. We also derive optimal step sizes for each scheme, such that the total error is minimized. Some of the schemes have the nice property that this step size also minimizes the variance of the error.Based on these results we show that to obtain good gradient estimates for noisy functions it is worthwhile to use DoE schemes. We recommend to implement such schemes in NLP solvers.

Keywords: Design of experiments, finite differences, gradient estimation, noisy functions

Suggested Citation

Brekelmans, R.C.M. and Driessen, Lonneke and Hamers, Herbert and den Hertog, Dick, Gradient Estimation Schemes for Noisy Functions (February 2003). CentER Working Paper No. 2003-12. Available at SSRN: https://ssrn.com/abstract=550121

R.C.M. Brekelmans (Contact Author)

Tilburg University - Tilburg University School of Economics and Management ( email )

P.O. Box 90153
B 1029
5000 LE Tilburg
Netherlands
+31 13 466 8234 (Phone)

Lonneke Driessen

Centre for Quantitative Methods ( email )

P.O. Box 414
5600 AK Eindhoven
Netherlands

Herbert Hamers

Tilburg University - Center for Economic Research (CentER) ( email )

P.O. Box 90153
Tilburg, 5000 LE
Netherlands
+31 13 4666 2660 (Phone)

Tilburg University - Department of Econometrics & Operations Research

Tilburg, 5000 LE
Netherlands

Dick Den Hertog

Tilburg University - Department of Econometrics & Operations Research ( email )

Tilburg, 5000 LE
Netherlands

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
496
PlumX Metrics