Getting it Right the First Time: Belief Elicitation with Novice Participants

Abstract

The auction design literature makes clear that theoretically equivalent mechanisms can perform very differently in practice. Though of equal importance, much less is known about the empirical performance of theoretically equivalent mechanisms for belief elicitation. This is especially unfortunate given the increasing interest in eliciting beliefs from (often novice) respondents in large-scale surveys. Using laboratory experiments with novice participants endowed with heterogeneous beliefs, we compare the empirical merit of two belief elicitation mechanisms proposed by Karni (2009), which we denote as "declarative" and "clock." These mechanisms are of interest because incentive compatibility does not require strong assumptions such as risk neutrality or expected utility maximization. Our key findings are that under the clock mechanism, (i) subjects are more likely to report their beliefs truthfully; and (ii) the distribution of elicited beliefs more accurately characterizes the true belief distribution. Our findings have substantial practical value to anyone wishing to elicit beliefs from novice respondents, a goal of increasing importance to large-scale survey design.

Keywords: probabilistic belief elicitation, declarative mechanism, clock mechanism, proper scoring rules, laboratory experiment

JEL Classification: C91