Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

50 Pages Posted: 11 Feb 2021

See all articles by Thomas Phelan

Thomas Phelan

Federal Reserve Banks - Federal Reserve Bank of Cleveland

Keyvan Eslami

Ryerson University

Date Written: February 10, 2021

Abstract

In this paper we explore some of the benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems. We first show that the implicit finite-difference scheme of Achdou et al. (2017) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate the benefits of departing from policy function iteration by showing that using variations of modified policy function iteration to solve income fluctuation problems in two and three dimensions can lead to an increase in the speed of convergence of more than an order of magnitude. We then show that the MCA method is also well-suited to solving portfolio problems with highly correlated state variables, a setting that commonly occurs within general equilibrium models with financial frictions and for which it is difficult to construct monotone (and hence convergent) finite-difference schemes.

Keywords: dynamic programming, financial frictions

JEL Classification: C63, E00, G11

Suggested Citation

Phelan, Thomas and Eslami, Keyvan, Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics (February 10, 2021). FRB of Cleveland Working Paper No. 21-04, Available at SSRN: https://ssrn.com/abstract=3783464 or http://dx.doi.org/10.2139/ssrn.3783464

Thomas Phelan (Contact Author)

Federal Reserve Banks - Federal Reserve Bank of Cleveland ( email )

East 6th & Superior
Cleveland, OH 44101-1387
United States

HOME PAGE: http://https://www.clevelandfed.org/our-research/economists/tom-phelan.aspx

Keyvan Eslami

Ryerson University ( email )

350 Victoria Street
Toronto, Ontario M5B 2K3
Canada

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
17
Abstract Views
94
PlumX Metrics