The Revealed Confidence Algorithm: Leveraging Advice-taking to Identify Experts and Improve Wisdom of Crowds

69 Pages Posted: 29 Jan 2021 Last revised: 28 Sep 2023

See all articles by Yunhao Zhang

Yunhao Zhang

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Date Written: November 29, 2020

Abstract

Identifying the experts within a crowd may help further improve the wisdom of crowds, especially when the crowd is biased. We propose a novel Revealed Confidence (RC) algorithm that employs the “RC measure” to better reflect the relative expertise of each agent in a crowd. This RC measure is the scaled amount of one’s belief-updating in response to numerical advice. The intuition, which we confirm both theoretically and empirically, is that those who are less swayed by numerical advice (e.g., the group mean) tend to be more accurate in their initial judgment. Therefore, taking a weighted average over the initial judgments with weights inversely proportional to the respective RC measures would place greater weight on the more accurate initial judgments in the aggregation. Consequently, the RC algorithm outperforms the conventional wisdom-of-crowds methods by overweighting the experts’ judgments. Moreover, we find a causal relationship between the amount of information one has and the degree of advice-taking in an experimental setting – experts (agents who have more information) are less swayed by numerical advice. Nevertheless, a higher stated confidence (e.g., a narrower confidence interval) does not correspond to having more information or being an expert. Since expertise is measured by the amount of information one has (and has not) taken into account, the RC algorithm may effectively distinguish the experts and improve Wisdom of Crowds, even when the stated confidence interval fails to do so.

Keywords: Judgment Aggregation, Wisdom of Crowds, Subjective uncertainty, Advice-taking, Belief-updating

JEL Classification: D79, D89

Suggested Citation

Zhang, Yunhao, The Revealed Confidence Algorithm: Leveraging Advice-taking to Identify Experts and Improve Wisdom of Crowds (November 29, 2020). Available at SSRN: https://ssrn.com/abstract=3739192 or http://dx.doi.org/10.2139/ssrn.3739192

Yunhao Zhang (Contact Author)

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

100 Main Street
E62-416
Cambridge, MA 02142
United States

HOME PAGE: http://https://mitsloan.mit.edu/phd/students/yunhao-zhang

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
181
Abstract Views
901
Rank
275,477
PlumX Metrics