Is It Better To Elicit Quantile Or Probability Judgments to Estimate a Continuous Distribution?

Abstract

Managers frequently rely on the judgment of an expert to estimate a probability distribution for a continuous random variable. Two elicitation methods are commonly used to gather this information: (i) quantile judgments for a set of fixed probability values, or (ii) cumulative probability judgments for a set of fixed variable values, but a consensus on which format yields more accurate distribution estimates has not been reached. We report results from a series of experiments conducted with participants with a range of quantitative backgrounds to compare these elicitation formats for a variety of variables, including synthetically generated numbers displayed in a video, daily high and low temperatures, the ages of people in the U.S. with a given name, commute times, home prices, and household incomes. We find that, although the two elicitation formats returned individual points with similar calibration and error magnitudes, quantile judgments provided more accurate estimates of the full distribution. We show that this relative advantage can be largely explained by the control afforded by the quantile elicitation format to place judgments in appropriately-spaced-apart positions on an individual’s subjective probability distribution.

Keywords: Elicitation, Subjective Probability Judgments, Distribution Estimation

JEL Classification: D83, C53, C91