Bias-Corrected Least-Squares Monte Carlo for Utility Based Optimal Stochastic Control Problems

32 Pages Posted: 14 Jun 2017 Last revised: 9 Apr 2019

See all articles by Johan Andreasson

Johan Andreasson

University of Technology Sydney (UTS); CSIRO Australia

Pavel V. Shevchenko

Macquarie University; Macquarie University, Macquarie Business School

Multiple version iconThere are 2 versions of this paper

Date Written: April 6, 2019

Abstract

The Least-Squares Monte Carlo (LSMC) method has gained popularity in recent years due to its ability to handle multi-dimensional stochastic control problems, including problems with state variables affected by control. However, when applied to the stochastic control problems in the multi-period expected utility models, the regression fit tends to contain errors which accumulate over time and typically blow up the numerical solution. In this paper we propose to transform the value function of the problems to improve the regression fit, and then using either the smearing estimate or smearing estimate with controlled heteroskedasticity to avoid the re-transformation bias in the estimates of the conditional expectations calculated in the LSMC algorithm. We also present and utilise recent improvements in the LSMC algorithms such as control randomisation with policy iteration to avoid accumulation of regression errors over time. Presented numerical examples demonstrate that transformation method leads to an accurate solution. In addition, in the forward simulation stage of the control randomisation algorithm, we propose a re-sampling of the state and control variables in their full domain at each time t and then simulating corresponding state variable at t+1, to improve the exploration of the state space that also appears to be critical to obtain a stable and accurate solution for the expected utility models.

Keywords: Dynamic Programming, Stochastic Control, Optimal Policy, Lifecycle Modelling

JEL Classification: D91, G11, C61

Suggested Citation

Andreasson, Johan and Shevchenko, Pavel V., Bias-Corrected Least-Squares Monte Carlo for Utility Based Optimal Stochastic Control Problems (April 6, 2019). Available at SSRN: https://ssrn.com/abstract=2985828 or http://dx.doi.org/10.2139/ssrn.2985828

Johan Andreasson (Contact Author)

University of Technology Sydney (UTS) ( email )

15 Broadway, Ultimo
PO Box 123
Sydney, NSW 2007
Australia

CSIRO Australia ( email )

41 Boggo Rd
Dutton Park, Queensland
Australia

Pavel V. Shevchenko

Macquarie University ( email )

North Ryde
Sydney, New South Wales 2109
Australia

HOME PAGE: http://www.businessandeconomics.mq.edu.au/contact_the_faculty/all_fbe_staff/pavel_shevchenko

Macquarie University, Macquarie Business School ( email )

New South Wales 2109
Australia

Register to save articles to
your library

Register

Paper statistics

Downloads
42
Abstract Views
313
PlumX Metrics