What Makes a Judgment 'Liberal'? Coding Bias in the United States Supreme Court Judicial Database

Abstract

This paper analyzes the validity of the widely used liberal/conservative judgment and vote "direction" variables reported in the United States Supreme Court Judicial Database (USSCJD). Leveraging information about a subset of cases in the USSCJD, namely those involving the constitutional review of congressional statutes, the paper shows that the USSCJD judgment variable is significantly more responsive to information highlighted in the Court's opinion, such as the identity of the majority opinion author, and significantly less responsive to factors not as conspicuously featured in the Court's opinion, such as the identity of the median Justice, or the composition of congressional preferences, relative to a broad range of alternative measures of the ideology of the Court's judgments. These findings are consistent with the presence of significant measurement bias in the USSCJD.