Managing Intelligence: Skilled Experts and Ai in Markets for Complex Products

54 Pages Posted: 28 Apr 2020 Last revised: 19 May 2023

See all articles by Jonathan Gruber

Jonathan Gruber

Massachusetts Institute of Technology (MIT) - Department of Economics

Benjamin Handel

University of California, Berkeley; National Bureau of Economic Research (NBER)

Samuel Kina

affiliation not provided to SSRN

Jonathan Kolstad

University of California, Berkeley - Haas School of Business; National Bureau of Economic Research (NBER)

Date Written: April 2020

Abstract

In numerous high stakes markets skilled experts play a key role in facilitating consumer choice of complex products. New artificial intelligence (AI) technologies are increasingly being used to augment expert decisions. We study the role of technology and expertise in the market for health insurance, where consumer choices are widely known to be sub-optimal. Our analysis leverages the large-scale implementation of an AI-based decision support tool in a private Medicare exchange where consumers are randomized to skilled agents over time. We find that, prior to AI-based technology, skilled experts in this market exhibit the same type of inconsistent behavior found in previous studies of individual choices, costing consumers $1260 on average. The addition of AI-based decision support improves outcomes by $278 on average and substantially reduces heterogeneity in broker performance. Experts efficiently synthesize private information, incorporating AI-based recommendations along dimensions that are well suited to AI (e.g. total expected patient costs), but overruling AI-based recommendations along dimensions for which humans are better suited (e.g. specifics of doctor networks). As a result, switching plans, an ex-post measure of plan satisfaction, is meaningfully lower for agents making AI-based recommendations. While AI is a complement to skill on average, we find that it is a substitute across the skill distribution; lower quality agents provide better recommendations with AI than the top agents did without it. Overall productivity rises, with the introduction of decision support associated with a 21% reduction in call time for enrollment.

Suggested Citation

Gruber, Jonathan and Handel, Benjamin and Kina, Samuel and Kolstad, Jonathan, Managing Intelligence: Skilled Experts and Ai in Markets for Complex Products (April 2020). NBER Working Paper No. w27038, Available at SSRN: https://ssrn.com/abstract=3586170

Jonathan Gruber (Contact Author)

Massachusetts Institute of Technology (MIT) - Department of Economics ( email )

50 Memorial Drive
E52-391
Cambridge, MA 02142
United States

Benjamin Handel

University of California, Berkeley ( email )

310 Barrows Hall
Berkeley, CA 94720
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Samuel Kina

affiliation not provided to SSRN

Jonathan Kolstad

University of California, Berkeley - Haas School of Business ( email )

545 Student Services Building, #1900
2220 Piedmont Avenue
Berkeley, CA 94720
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
41
Abstract Views
361
PlumX Metrics