Measuring the Impact of AI on Information Worker Productivity

6 Pages Posted: 1 Dec 2023

See all articles by Benjamin G. Edelman

Benjamin G. Edelman

Microsoft Corporation

Donald Ngwe

Microsoft Corporation

Sida Peng

Microsoft Corporation

Date Written: November 29, 2023

Abstract

This paper reports the results of two randomized controlled trials evaluating the performance and user satisfaction of a new AI product in the context of common information worker tasks. We designed workplace scenarios to test common information worker tasks: retrieving information from files, emails, and calendar; catching up after a missed online meeting; and drafting prose. We assigned these tasks to 310 subjects tasked to find relevant information, answer multiple choice questions about what they found, and write marketing content. In both studies, users with the AI tool were statistically significantly faster, a difference that holds both on its own and when controlling for accuracy/quality. Furthermore, users who tried the AI tool reported higher willingness to pay relative to users who merely heard about it but didn't get to try it, indicating that the product exceeded expectations.

Keywords: Artificial Intelligence, Productivity

JEL Classification: O32, L86, J24, C93

Suggested Citation

Edelman, Benjamin G. and Ngwe, Donald and Peng, Sida, Measuring the Impact of AI on Information Worker Productivity (November 29, 2023). Available at SSRN: https://ssrn.com/abstract=4648686 or http://dx.doi.org/10.2139/ssrn.4648686

Benjamin G. Edelman (Contact Author)

Microsoft Corporation ( email )

One Microsoft Way
Redmond, WA 98052
United States

HOME PAGE: http://www.benedelman.org/

Donald Ngwe

Microsoft Corporation ( email )

One Microsoft Way
Redmond, WA 98052
United States

Sida Peng

Microsoft Corporation ( email )

One Microsoft Way
Redmond, WA 98052
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
596
Abstract Views
1,239
Rank
87,094
PlumX Metrics