Time Consistent Reinforcement Learning for Optimal Consumption Under Epstein-Zin Preferences

34 Pages Posted: 20 Mar 2023

See all articles by Matthew Francis Dixon

Matthew Francis Dixon

Illinois Institute of Technology

Ivan Gvozdanovic

Illinois Institute of Technology

Dominic O'Kane

EDHEC Business School - EDHEC Risk Climate Impact Institute

Date Written: March 14, 2023

Abstract

We present a class of least squares reinforcement learning algorithms for optimal consumption under elasticity of intertemporal substitution and risk aversion preferences. The classical setting of Epstein-Zin utility preferences is cast into a dynamic utility functional framework and shown to exhibit time consistency. As a dynamic utility function, we find the robust approximation of the optimal consumption problem as a discrete time Markov Decision Process. We present a least-squares Q-Learning algorithm suitable for non-linear monotone certainty equivalents and benchmark its policy estimation convergence properties on an optimal wealth consumption problem against Least Squares Monte-Carlo and binomial tree methods. Finally, we demonstrate our least-squares Q-learning algorithm on an optimal consumption problem applied to SPDR S&P 500 ETF Trust (SPY) data.

Keywords: Optimal Consumption, Dynamic Utility Theory, Certainty Equivalents, Reinforcement Learning, Time consistency, Epstein-Zin, Wealth Management

Suggested Citation

Dixon, Matthew Francis and Gvozdanovic, Ivan and O'Kane, Dominic, Time Consistent Reinforcement Learning for Optimal Consumption Under Epstein-Zin Preferences (March 14, 2023). Available at SSRN: https://ssrn.com/abstract=4388762 or http://dx.doi.org/10.2139/ssrn.4388762

Matthew Francis Dixon (Contact Author)

Illinois Institute of Technology ( email )

Department of Mathematics
W 32nd St., E1 room 208, 10 S Wabash Ave, Chicago,
Chicago, IL 60616
United States

Ivan Gvozdanovic

Illinois Institute of Technology ( email )

10 W 35th St
Chicago, IL 60616
United States

Dominic O'Kane

EDHEC Business School - EDHEC Risk Climate Impact Institute ( email )

United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
151
Abstract Views
488
Rank
377,199
PlumX Metrics