Optimal Stopping for Medical Treatment with Predictive Information
48 Pages Posted: 13 Jun 2019 Last revised: 13 Jan 2020
Date Written: June 1, 2019
Data availability and advancement in machine learning techniques make accurate prediction of the future a foreseeable reality. How to efficiently utilize the predictive information in a multistage medical decision-making environment, however, remains understudied. In this paper, we develop a discrete-time, finite-horizon Markov decision process model, incorporating perfect predictive information, to support decisions on medical treatment continuation. We extend our framework to a situation with prediction errors, using a partially observable Markov decision process. We characterize the structure of the optimal policies under both settings and show that knowing predictive information can lead to significantly different decision protocols. We calibrate and test our models with an extubation problem in an intensive care unit (ICU). Using a patient-level data set, we compare the performance of different extubation policies and demonstrate that incorporating predictive information can decrease extubation failure rate and reduce ICU length-of-stay of ventilated patients, especially for patients with poor initial conditions.
Keywords: Medical Decision Making; Optimal Stopping; Predictive Information; Partially Observable Markov Decision Process; Intensive Care Unit
Suggested Citation: Suggested Citation