Language Model Interpretability and Empirical Legal Studies

Virginia Public Law and Legal Theory Research Paper No. 2023-69

Forthcoming Journal of Institutional and Theoretical Economics

36 Pages Posted: 12 Oct 2023

See all articles by Michael A. Livermore

Michael A. Livermore

University of Virginia School of Law

Felix Herron

Sorbonne University

Daniel Rockmore

Dartmouth College - Department of Mathematics; Dartmouth College - Department of Computer Science

Date Written: October 11, 2023

Abstract

Large language models (LLMs) now perform extremely well on many natural language processing tasks. Their ability to convert legal texts to data may offer empirical legal studies (ELS) scholars a low-cost alternative to research assistants in many contexts. However, less complex computational language models, such as topic modeling and sentiment analysis, are more interpretable than LLMs. In this paper we highlight these differences by comparing LLMs with less complex models on three ELS-related tasks. Our findings suggest that ELS research will—for the time being—benefit from combining LLMs with other techniques to optimize the strengths of each approach.

Keywords: empirical legal studies; natural language processing; interpretability; language models; computational analysis of law; law-as-data

Suggested Citation

Livermore, Michael A. and Herron, Felix and Rockmore, Daniel, Language Model Interpretability and Empirical Legal Studies (October 11, 2023). Virginia Public Law and Legal Theory Research Paper No. 2023-69, Forthcoming Journal of Institutional and Theoretical Economics, Available at SSRN: https://ssrn.com/abstract=4599212

Michael A. Livermore (Contact Author)

University of Virginia School of Law ( email )

Felix Herron

Sorbonne University

Daniel Rockmore

Dartmouth College - Department of Mathematics ( email )

United States

Dartmouth College - Department of Computer Science ( email )

United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
150
Abstract Views
464
Rank
357,072
PlumX Metrics