Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology

126 Pages Posted: 10 Jul 2023 Last revised: 24 Jul 2023

See all articles by Nikhil Agarwal

Nikhil Agarwal

Massachusetts Institute of Technology (MIT) - Department of Economics; Yale University - Cowles Foundation

Alex Moehring

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Pranav Rajpurkar

Harvard University - Department of Biomedical Informatics

Tobias Salz

Massachusetts Institute of Technology (MIT); National Bureau of Economic Research (NBER)

Date Written: July 2023

Abstract

Full automation using Artificial Intelligence (AI) predictions may not be optimal if humans can access contextual information. We study human-AI collaboration using an information experiment with professional radiologists. Results show that providing (i) AI predictions does not always improve performance, whereas (ii) contextual information does. Radiologists do not realize the gains from AI assistance because of errors in belief updating – they underweight AI predictions and treat their own information and AI predictions as statistically independent. Unless these mistakes can be corrected, the optimal human-AI collaboration design delegates cases either to humans or to AI, but rarely to AI assisted humans.

Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Suggested Citation

Agarwal, Nikhil and Moehring, Alex and Rajpurkar, Pranav and Salz, Tobias, Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology (July 2023). NBER Working Paper No. w31422, Available at SSRN: https://ssrn.com/abstract=4505053

Nikhil Agarwal (Contact Author)

Massachusetts Institute of Technology (MIT) - Department of Economics ( email )

50 Memorial Drive
E52-391
Cambridge, MA 02142
United States

Yale University - Cowles Foundation ( email )

Box 208281
New Haven, CT 06520-8281
United States

Alex Moehring

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

100 Main Street
E62-416
Cambridge, MA 02142
United States

Pranav Rajpurkar

Harvard University - Department of Biomedical Informatics ( email )

Tobias Salz

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
8
Abstract Views
427
PlumX Metrics