The Impact of Prediction Mode on Forecasting Accuracy: The Role of Mental Models

Abstract

Forecasters predicting how people change their behavior in response to a treatment
or participating in the intervention often consider a set of alternatives. In
contrast, those who are treated are typically exposed to only one of the treatment
alternatives. For example, managers selecting a wage schedule consider a set of alternative
wages while employees are hired at a given rate. We show that forecasts
made in Joint-prediction mode—which considers a set of alternatives—generate
predictions that expect substantially larger behavioral responses than those made
in Separate-prediction mode—which considers the response to only one treatment
realization in isolation. Results show the latter to be more accurate in matching
people’s actual responses to interventions and treatment changes. Our findings suggest
that the discrepancy in accuracy is due to a disparity in the mental models
used by forecasters and those being treated. We present applications to managerial
decision-making and forecasting of scientific results.

Keywords: Forecasting, Choice under Uncertainty, Prediction