The Role of Feedback in Dynamic Crowdsourcing Contests: A Structural Empirical Analysis

Abstract

In this paper, we empirically examine the impact of performance feedback on the outcome of crowdsourcing contests. We develop a dynamic structural model to capture the economic processes that drive contest participants' behavior, and estimate the model using a detailed data set about real online logo design contests. Our rich model captures key features of the crowdsourcing context, including a large participant pool, entries by new participants throughout the contest, exploitation (revision of previous submissions) and exploration (radically novel submissions) behaviors by contest incumbents, and the participants' strategic choice among these entry, exploration, and exploitation decisions in a dynamic game. Using counter-factual simulations, we compare the outcome of crowdsourcing contests under alternative feedback disclosure policies and award levels. Our simulation results suggest that despite its prevalence on many platforms, the full feedback policy (providing feedback throughout the contest) may not be optimal. The late feedback policy (providing feedback only in the second half of the contest) leads to a better overall contest outcome.

Keywords: Crowdsourcing Contests, Feedback, Econometric Analysis, Structural Modeling, Dynamic Game