Leveraging Advice-taking and Kernel Density Estimation to Identify A Cluster of Experts and Improve Wisdom of Crowds

39 Pages Posted: 15 Apr 2024

See all articles by Yunhao Zhang

Yunhao Zhang

Massachusetts Institute of Technology (MIT) - Sloan School of Management; University of California, Berkeley - Haas School of Business

Date Written: November 4, 2023

Abstract

Wisdom of Crowds estimates (e.g., a group mean) can suffer when non-experts in the crowd have biased estimations. One solution is to aggregate over a subset of experts instead of the entire crowd -- but systematically identifying the accurate individuals remains a challenge in a single-prediction-problem context. We introduce a novel algorithm to identify experts in the crowd by combining kernel density estimation and the behavioral metric “weight on advice” (WOA) to identify and then aggregate a distinct cluster of accurate estimates. This approach is based on two phenomena: First, accurate experts revise their initial estimates less upon seeing numerical advice compared to their non-expert counterparts; and second, while experts' estimates tend to cluster close to the truth, stubborn non-experts may have a wide range of wrong answers without forming their own cluster. Therefore, although relying solely on WOA or cluster size may not identify experts (due to the existence of stubborn non-experts or the largest cluster being formed by an overwhelming number of non-experts), we demonstrate that finding and aggregating over the cluster that has the least average WOA does, thereby improving crowd wisdom. Our theoretical framework not only demonstrates the algorithm's rational foundation, but also characterizes its properties accounting for the presence of agents who exhibit stubbornness and over-updating in advice-taking. Crucially, our empirical analysis, combined with open data from other publications examining various contexts, confirms that averaging over the (updated) judgment within our identified cluster outperforms averaging over the initial or updated judgment of the entire group.

Keywords: Wisdom of Crowds, Judgment Aggregation, Forecasting, Advice-taking, Identify Experts

Suggested Citation

Zhang, Yunhao, Leveraging Advice-taking and Kernel Density Estimation to Identify A Cluster of Experts and Improve Wisdom of Crowds (November 4, 2023). MIT Sloan Research Paper Forthcoming, Available at SSRN: https://ssrn.com/abstract=4779145 or http://dx.doi.org/10.2139/ssrn.4779145

Yunhao Zhang (Contact Author)

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

100 Main Street
E62-416
Cambridge, MA 02142
United States

HOME PAGE: http://https://mitsloan.mit.edu/phd/students/yunhao-zhang

University of California, Berkeley - Haas School of Business ( email )

545 Student Services Building, #1900
2220 Piedmont Avenue
Berkeley, CA 94720
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
33
Abstract Views
101
PlumX Metrics