Understanding People's Preferences for Predictions: People Prioritize Being Right over Minimizing How Wrong They Are in Expectation

Abstract

This work explores the preferences that laypeople exhibit when making and evaluating predictions in the form of point estimates (e.g., the high temperature will be 66°). I propose that people typically have diminishing sensitivity to prediction error – the absolute difference between a prediction and a realized outcome. As a result, people often prioritize “being right" – focusing on achieving near perfect predictions and placing less emphasis on the magnitude of errors when errors occur. Across 16 studies using varying methods and stimuli, participants exhibited multiple behaviors consistent with diminishing sensitivity to prediction error: (i) predicting the mode of distributions, (ii) restricting predictions to possible outcomes, (iii) reporting decreasing reactions to increasing marginal units of error, and (iv) preferring predictive models built with diminishing sensitivity to error. This behavior diverges from traditional methods of building predictive models and common interpretations of people’s predictions, which often prioritize avoiding large errors and assume that people are predicting the mean. Ultimately, this work not only highlights the discrepancies between our current practices and people’s preferences for predictions, but also calls for a more thorough exploration of human objectives before we build models for them to use or make inferences about their beliefs in light of a decision they made.

Keywords: Point estimate, Judgment, Decision making, Algorithm, Forecasting, Modeling