Algorithmic Decision-Making Safeguarded by Human Knowledge

49 Pages Posted: 22 Nov 2022

See all articles by Ningyuan Chen

Ningyuan Chen

University of Toronto - Rotman School of Management

Ming Hu

University of Toronto - Rotman School of Management

Wenhao Li

University of Toronto - Rotman School of Management

Date Written: November 17, 2022

Abstract

Commercial AI solutions provide analysts and managers with data-driven business intelligence for a wide range of decisions, such as demand forecasting and pricing. However, human analysts may have their own insights and experiences about the decision-making that is at odds with the algorithmic recommendation. In view of such a conflict, we provide a general analytical framework to study the augmentation of algorithmic decisions with human knowledge: the analyst uses the knowledge to set a guardrail by which the algorithmic decision is clipped if the algorithmic output is out of bound, and seems unreasonable. We study the conditions under which the augmentation is beneficial relative to the raw algorithmic decision. We show that when the algorithmic decision is asymptotically optimal with large data, the non-data-driven human guardrail usually provides no benefit. However, we point out three common pitfalls of the algorithmic decision: (1) lack of domain knowledge, such as the market competition, (2) model misspecification, and (3) data contamination. In these cases, even with sufficient data, the augmentation from human knowledge can still improve the performance of the algorithmic decision.

Keywords: algorithmic decision, human knowledge, model misspecification, data contamination

Suggested Citation

Chen, Ningyuan and Hu, Ming and Li, Wenhao, Algorithmic Decision-Making Safeguarded by Human Knowledge (November 17, 2022). Rotman School of Management Working Paper No. 4281867, Available at SSRN: https://ssrn.com/abstract=4281867 or http://dx.doi.org/10.2139/ssrn.4281867

Ningyuan Chen

University of Toronto - Rotman School of Management ( email )

Ming Hu

University of Toronto - Rotman School of Management ( email )

105 St. George st
Toronto, ON M5S 3E6
Canada
416-946-5207 (Phone)

HOME PAGE: http://ming.hu

Wenhao Li (Contact Author)

University of Toronto - Rotman School of Management ( email )

105 St. George Street
Toronto, Ontario M5S 3E6 M5S1S4
Canada

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
96
Abstract Views
398
Rank
466,440
PlumX Metrics