The Revealed Expertise Algorithm: Leveraging Advice-taking to Identify Experts and Improve Wisdom of Crowds

59 Pages Posted: 29 Jan 2021 Last revised: 15 Aug 2022

See all articles by Yunhao Zhang

Yunhao Zhang

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Date Written: November 29, 2020

Abstract

Identifying the experts within a crowd may help further improve the wisdom of crowds. We propose a new Revealed Expertise (RE) algorithm that uses the "RE measure", which is a scaled amount of belief updating given numerical advice (i.e., the group mean), as a proxy for prior variance to better reflect the relative expertise of each agent in a crowd. The intuition, which we confirm empirically, is that those who are less swayed by the group mean tend to be more accurate in their initial judgment. Therefore, using inverse-variance weighting with the RE measures as the variance inputs improves upon the existing wisdom-of-crowd methods by over-weighting the more accurate initial judgments in the aggregation. Crucially, we demonstrate that advice-taking is able to reveal the amount of information one has and has not taken into account in their initial judgment, even when self-reported confidence mistakes the less informed as the experts. In addition, we propose a pre-registered method in which we measure subjects' bias in advice-taking to calibrate the RE measures and further improve the algorithm's performance.

Keywords: Judgment Aggregation, Wisdom of Crowds, Uncertainty, Belief-update

JEL Classification: D79, D89

Suggested Citation

Zhang, Yunhao, The Revealed Expertise Algorithm: Leveraging Advice-taking to Identify Experts and Improve Wisdom of Crowds (November 29, 2020). Available at SSRN: https://ssrn.com/abstract=3739192 or http://dx.doi.org/10.2139/ssrn.3739192

Yunhao Zhang (Contact Author)

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

100 Main Street
E62-416
Cambridge, MA 02142
United States

HOME PAGE: http://https://mitsloan.mit.edu/phd/students/yunhao-zhang

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
104
Abstract Views
528
rank
354,073
PlumX Metrics