Lawyering in the Age of Artificial Intelligence

64 Pages Posted: 9 Nov 2023 Last revised: 24 Feb 2024

See all articles by Jonathan H. Choi

Jonathan H. Choi

University of Southern California; University of Southern California Gould School of Law

Amy Monahan

University of Minnesota Law School

Daniel Schwarcz

University of Minnesota Law School

Date Written: November 7, 2023

Abstract

We conducted the first randomized controlled trial to study the effect of AI assistance on human legal analysis. We randomly assigned law school students to complete realistic legal tasks either with or without the assistance of GPT-4. We tracked how long the students took on each task and blind-graded the results. We found that access to GPT-4 only slightly and inconsistently improved the quality of participants’ legal analysis but induced large and consistent increases in speed. AI assistance improved the quality of output unevenly—where it was useful at all, the lowest-skilled participants saw the largest improvements. On the other hand, AI assistance saved participants roughly the same amount of time regardless of their baseline speed. In follow up surveys, participants reported increased satisfaction from using AI to complete legal tasks and correctly predicted the tasks for which GPT-4 were most helpful. These results have important descriptive and normative implications for the future of lawyering. Descriptively, they suggest that AI assistance can significantly improve productivity and satisfaction, and that they can be selectively employed by lawyers in areas where they are most useful. Because these tools have an equalizing effect on performance, they may also promote equality in a famously unequal profession. Normatively, our findings suggest that law schools, lawyers, judges, and clients should affirmatively embrace AI tools and plan for a future in which they will become widespread.

Suggested Citation

Choi, Jonathan H. and Monahan, Amy and Schwarcz, Daniel, Lawyering in the Age of Artificial Intelligence (November 7, 2023). Minnesota Law Review, Forthcoming, Minnesota Legal Studies Research Paper No. 23-31, Available at SSRN: https://ssrn.com/abstract=4626276 or http://dx.doi.org/10.2139/ssrn.4626276

Jonathan H. Choi

University of Southern California ( email )

2250 Alcazar Street
Los Angeles, CA 90089
United States

University of Southern California Gould School of Law ( email )

699 Exposition Blvd.
Los Angeles, CA 90089
United States

Amy Monahan

University of Minnesota Law School ( email )

229 19th Avenue South
Minneapolis, MN 55455
United States

Daniel Schwarcz (Contact Author)

University of Minnesota Law School ( email )

229 19th Avenue South
Minneapolis, MN 55455
United States

HOME PAGE: http://www.law.umn.edu/profiles/daniel-schwarcz

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
6,356
Abstract Views
28,163
Rank
2,226
PlumX Metrics