A Cautious Multi-Advisor Sequential Decision-Making Strategy Without Ground Truth for Maximizing the Profits

Abstract

Multi-advisor decision-making without access to ground truth presents a significant challenge across various domains, such as lending, investment, ensemble learning, and crowdsourcing. In such scenarios, each decision option carries distinct values, and querying advisors incurs costs. The challenge lies in designing an advisor selection and decision-making strategy that retrieves reliable advice while maximizing decision profits. In this paper, we propose a novel approach, Multi-Advisor Decision Making for Maximizing Profits, to address the challenge above. Our contributions are as follows: First, we propose a novel trustworthiness model that provides a precise assessment of each advisor’s trustworthiness without relying on ground truth. Second, we introduce an adaptive advisor selection strategy based on balancing advisor costs, trustworthiness, and decision option values. Third, our cautious decision model adaptively adjusts decision-making strategies based on the uncertainty in advisors’ trustworthiness, achieving high decision accuracy. Additionally, we conduct extensive experiments to evaluate our method. The results show that our method significantly outperforms eleven baselines across twelve conditions.

Keywords: Group Decision-Making, Crowdsourcing, Truth inference