Limited Dynamic Forecasting of Hidden Markov Models

64 Pages Posted: 13 Jul 2018

See all articles by Arun Gopalakrishnan

Arun Gopalakrishnan

Rice University - Jones Graduate School of Business

Eric Bradlow

University of Pennsylvania - Marketing Department

Peter Fader

University of Pennsylvania - Marketing Department

Date Written: July 1, 2018

Abstract

Hidden Markov Models (HMMs) have emerged as an empirical “workhorse” in the marketing literature in capturing and forecasting within-customer non-stationary behaviors. Extant research has demonstrated that HMMs typically outperform nested benchmarks when examining fit statistics aggregated over individuals and time, but have remained largely silent on the set of dynamic out-of-sample forecasting paths offered by an HMM at the individual level. We examine the capabilities of a two-state HMM using theory and reveal a surprising result: an HMM’s forecasting paths are generally limited to monotonic mean-reverting trajectories. Specifically, they lack the notable flexibility associated with the in-sample state-switching imputations, which are generally (but, as we show, erroneously) presumed to exist in the holdout sample as well. Further, we find that common HMM extensions such as adding more hidden states, allowing for heterogeneity, allowing for covariates, and using hidden semi-Markov models do not alleviate the limited forecasting flexibility. Using a simulation design, we show how these limitations can affect forecasting performance empirically. We discuss implications of the limited forecasting properties of HMMs for researchers and managers.

Keywords: Hidden Markov Model; Forecasting; Individual-level dynamics; Latent state trajectories

JEL Classification: M31, C11

Suggested Citation

Gopalakrishnan, Arun and Bradlow, Eric and Fader, Peter, Limited Dynamic Forecasting of Hidden Markov Models (July 1, 2018). Available at SSRN: https://ssrn.com/abstract=3206425 or http://dx.doi.org/10.2139/ssrn.3206425

Arun Gopalakrishnan (Contact Author)

Rice University - Jones Graduate School of Business ( email )

6100 South Main Street
P.O. Box 1892
Houston, TX 77005-1892
United States

Eric Bradlow

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-8255 (Phone)

Peter Fader

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
141
Abstract Views
819
rank
205,498
PlumX Metrics