Bias-Corrected Least-Squares Monte Carlo for Utility Based Optimal Stochastic Control Problems
30 Pages Posted: 20 Jun 2018 Last revised: 23 Dec 2018
Date Written: December 16, 2018
The Least-Squares Monte Carlo method has gained popularity recent years due to its ability to handle multi-dimensional stochastic control problems without restrictions on the state dynamics, including problems with state variables affected by control. However, when applied to stochastic control problems in the multiperiod expected utility models, the regression t tends to contain errors which accumulate over time and typically blow up the numerical solution. In this paper we propose to transform the value function of stochastic control problems to improve the regression t, and then using either the 'Smearing Estimate' or 'Controlled Heteroskedasticity' to avoid the re-transformation bias. We also present and utilise recent improvements in Least-Squares Monte Carlo algorithms such as control randomisation with policy iteration to avoid regression errors from accumulating. Presented numerical examples demonstrate that our transformation method allows for control of disturbance terms to be handled correctly and leads to an accurate solution. In addition, in the forward simulation stage of the algorithm, we propose a re-sampling of state variables at each time step instead of simulating continuous paths, to improve the exploration of the state space that also appears to be important to obtain a stable and accurate solution for expected utility models.
Keywords: Dynamic programming, Least-Squares Monte Carlo, control randomisation, stochastic control, lifecycle modelling
JEL Classification: D91, G11, C61
Suggested Citation: Suggested Citation