Detecting Inconsistency in Governance

36 Pages Posted: 24 Jul 2016 Last revised: 30 Jul 2018

See all articles by Ryan Copus

Ryan Copus

Harvard Law School

Ryan Hubert

University of California, Davis - Department of Political Science

Date Written: October 17, 2017


Measurements of inconsistency among decision makers are important for assessing the quality of governance, as well evaluating policy reforms. And yet, high quality measures of inconsistency in real-world institutions are rarely available. The core problem is that mean differences on outcomes between decision makers (disparity statistics) systematically understate inconsistency. Our contribution is twofold. First, we show how this downward bias can theoretically be eliminated by targeting a specific kind of heterogeneous treatment effect. Second, we demonstrate how machine learning can be used to optimally implement the theory on observational data. We leverage an original dataset of civil appeals in the Ninth Circuit to provide one of the first high quality measures of decision making inconsistency in the court.

Suggested Citation

Copus, Ryan and Hubert, Ryan, Detecting Inconsistency in Governance (October 17, 2017). Available at SSRN: or

Ryan Copus

Harvard Law School ( email )

1575 Massachusetts
Hauser 406
Cambridge, MA 02138
United States

Ryan Hubert (Contact Author)

University of California, Davis - Department of Political Science ( email )

One Shields Avenue
Davis, CA 95616
United States


Register to save articles to
your library


Paper statistics

Abstract Views
PlumX Metrics