Use of a Markov Decision Process Model for Treatment Selection in an Asymptomatic Disease with Consideration of Risk Sensitivity

Simon School Working Paper No. FR 11-10

Socio-Economic Planning Sciences 2012

29 Pages Posted: 21 Mar 2011 Last revised: 8 May 2013

See all articles by Vera Tilson

Vera Tilson

University of Rochester - Simon Business School

David Tilson

Simon Graduate School of Business, University of Rochester

Date Written: December 20, 2011

Abstract

Some potentially dangerous diseases are completely asymptomatic. Their diagnosis as incidental findings of ever more sensitive medical imaging can leave patients and physicians in something of a quandary. The patient feels well and potential interventions to stave off long term deterioration or death bring with them immediate risks. We discuss the use of a Markov Decision Process (MDP) model (rather than Monte Carlo simulation of a Markov Model) to create a tool for analyzing individual treatment decisions for asymptomatic chronic diseases where a patient's condition cannot improve.

We formulate a finite-horizon MDP model to determine optimal treatment plans and discuss three distinct optimality criteria: (a) maximizing expected quality-adjusted-life years with and without discounting, (b) maximizing the expected number of life years in good health, and (c) maximizing the expected utility for number of years in good health. In (c) we assume exponential utility and consider different risk aversion factors reported in the medical literature. We illustrate the model’s use by considering asymptomatic intracranial aneurysm. Our model builds on a simulation model (Takao & Noji, 2007) created to examine treatment recommendations based on cost-effectiveness. We demonstrate that incorporating risk aversion leads to “no treatment” recommendations for some types of aneurysm. Furthermore, the use of alternate patient-selected criteria leads to recommendations that vary from (Takao & Noji, 2007) in several scenarios. We also discuss the use of the software as a decision support tool to help make individualized treatment recommendations and demonstrate that the computational performance of the algorithm makes its use feasible during a short office visit.

Keywords: Markov Decision Process Model, Medical Decision Making, Decision Support System, Intracranial Aneurysm

JEL Classification: I19

Suggested Citation

Tilson, Vera and Tilson, David, Use of a Markov Decision Process Model for Treatment Selection in an Asymptomatic Disease with Consideration of Risk Sensitivity (December 20, 2011). Socio-Economic Planning Sciences 2012. Available at SSRN: https://ssrn.com/abstract=1718142 or http://dx.doi.org/10.2139/ssrn.1718142

Vera Tilson (Contact Author)

University of Rochester - Simon Business School ( email )

Rochester, NY 14627
United States

David Tilson

Simon Graduate School of Business, University of Rochester ( email )

Rochester, NY 14627
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
130
Abstract Views
1,104
rank
221,170
PlumX Metrics