Assessing AI Output in Legal Decision-Making with Nearest Neighbors

124 Penn State Law Review 2020, 609–655.

47 Pages Posted: 6 Oct 2019 Last revised: 7 Jul 2020

See all articles by Timothy Lau

Timothy Lau

Federal Judicial Center

Alex Biedermann

University of Lausanne - Faculty of Law, Criminal Justice and Public Administration

Date Written: September 12, 2019

Abstract

Artificial intelligence (“AI”) systems are widely used to assist or automate decision-making. However, while there are general metrics for performance of AI systems, there is as yet no well-established gauge to assess the quality of a particular AI recommendation or decision. This presents a serious problem in the emerging use of AI in legal applications because the legal system aims at good performance not only in the aggregate but also in individual cases. This article presents the concept of using nearest neighbors to assess individual AI output. The method has the benefit of being easy to understand and apply for judges, lawyers, and juries. In addition, it is fundamentally compatible with existing AI methodologies. The paper explains how the concept could be applied for probing AI output in a number of use cases, including civil discovery, risk prediction, and forensic comparison. It also presents the limitations of the concept.

Keywords: civil discovery, risk assessments, forensic science, decision-making, artificial intelligence

Suggested Citation

Lau, Timothy and Biedermann, Alex, Assessing AI Output in Legal Decision-Making with Nearest Neighbors (September 12, 2019). 124 Penn State Law Review 2020, 609–655., Available at SSRN: https://ssrn.com/abstract=3459870

Timothy Lau (Contact Author)

Federal Judicial Center ( email )

Charlottesville, VA 22902
United States

Alex Biedermann

University of Lausanne - Faculty of Law, Criminal Justice and Public Administration ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
73
Abstract Views
631
Rank
659,158
PlumX Metrics