An Expert-Sourced Measure of Judicial Ideology

Abstract

We introduce the first ideology measure covering every non-Supreme-Court Article III judge on a single scale. The dataset comprises dynamic, interval-level, and potentially multi-dimensional data of every federal district and appellate judge serving since 1985. The measure is derived from many thousands of qualitative evaluations by a representative sample of legal experts familiar with those judges’ approaches to judging. By drawing on expert evaluations, our method overcomes many of the issues that limit existing methods, including endogeneity, sample bias, lack of dynamic capability, and difficulty measuring the lower courts. Notably, our scores are perhaps the first to take account of the subtle differences in judges’ opinions and other law-making behavior, allowing them to differentiate between the vast majority of judges in the dataset. Our data also cover several times as many judges as the next largest dataset and are dynamic, include standard errors, and cover judicial traits that no existing measure captures. Analysis of a set of appellate-decision data indicates that our time-aggregated point estimates predict appellate outcomes more accurately than existing appellate-judge ideology measures.