Devising Rule of Law Baselines: The Next Step in Quantitative Studies of Judging

Abstract

Political scientists and law professors have lately taken to asserting that quantitative studies of judging reveal worrisome findings about the rule of law in the U.S. judicial system. The authors of Are Judges Political?, for example, declare: “We show that variations in panel composition lead to dramatically different outcomes, in a way that creates serious problems for the rule of law.”

To evaluate such assertions, one must first know what the rule of law requires of judges; then one must identify or measure how much, or in what ways, judges are falling short of these requirements: there must be rule of law baselines or standards. None exist, however - a gap which allows researchers to draw misleading conclusions from the results of their studies.

This essay demonstrates the need for rule of law baselines and offers several proposals for how they might be constructed (with due warnings about their limitations). Furthermore, it argues that the incorporation of such baselines into quantitative research on judging will enhance the value of the information produced in these studies.

This brief essay was presented at a two-day colloquium at Duke Law School involving federal judges, legal theorists, and political scientists, which was convened to facilitate a productive exchange of ideas on the design and orientation of quantitative research on judging.