Judgment Aggregation in Creative Production: Evidence from the Movie Industry

Abstract

This paper studies a novel, light-touch approach to aggregate judgment from a large number of industry experts on ideas that they encounter in their normal course of business. Our context is the movie industry, in which customer appeal is difficult to predict and investment costs are high. The Black List, an annual publication, ranks unproduced scripts based on anonymous nominations from film executives. While low participation costs enable a high response rate from busy executives, this results in a trade-off, as the nominations lack standard criteria, and which voters see which ideas may be unobservable and affected by various factors. Despite these challenges, we find that such an approach to aggregation is predictive: listed scripts are substantially more likely to be released than observably similar but unlisted scripts, and, conditional on release and investment levels, to generate higher box office revenues. We also find that the results differ by the writer’s experience level: (i) scripts from less-experienced writers are, in fact, more likely to be listed and to rank higher if listed; (ii) but even though being listed is associated with a higher release rate within less-experienced writers, the discrepancy in release probabilities relative to experienced writers remains large (even for top-ranked scripts). Both of these results can be explained by the idea that scripts from less-experienced writers are more visible among eligible voters.