Bounding Human Effort in Augmented Intelligence Systems

36 Pages Posted: 13 Jan 2025 Last revised: 12 Jan 2025

See all articles by Emaad Manzoor

Emaad Manzoor

Cornell University

Jordan Tong

Wisconsin School of Business

Sriniketh Vijayaraghavan

Wisconsin School of Business

Rui Li

Pinterest

Date Written: January 11, 2025

Abstract

Augmented intelligence systems leverage human-algorithm collaboration to improve decisionmaking in various settings. The performance of such systems hinges on both the algorithm's accuracy and the effort required by humans to correct the algorithm's mistakes. Yet, human correction effort has been ignored in the design of augmented intelligence systems thus far. In this work, we propose a framework that provably endows algorithms in augmented intelligence systems with bounds on the expected human effort required to correct their mistakes. We collaborate with a large social media firm to expand their advertising interest taxonomy using augmented intelligence, and show that bounding human correction effort (keeping algorithmic accuracy fixed) significantly improves the performance of the augmented intelligence system overall by increasing taxonomists' decision-making speed and accuracy. More generally, our research reveals a novel human-centric dimension of optimization for the imperfect algorithms commonly embedded in augmented intelligence systems.

Keywords: Human-Algorithm Hybrids, Augmented Intelligence, Ad Interest Taxonomies

Suggested Citation

Manzoor, Emaad and Tong, Jordan and Vijayaraghavan, Sriniketh and Li, Rui, Bounding Human Effort in Augmented Intelligence Systems (January 11, 2025). Cornell SC Johnson College of Business Research Paper, Available at SSRN: https://ssrn.com/abstract=5094232 or http://dx.doi.org/10.2139/ssrn.5094232

Emaad Manzoor (Contact Author)

Cornell University ( email )

Jordan Tong

Wisconsin School of Business ( email )

975 University Avenue
Madison, WI 53706
United States

Sriniketh Vijayaraghavan

Wisconsin School of Business ( email )

975 University Ave
4185
Madison, WI 53703
United States
3472055011 (Phone)

Rui Li

Pinterest ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
72
Abstract Views
268
Rank
690,722
PlumX Metrics