Examiner Inconsistency: Evidence from Refugee Appeals

90 Pages Posted: 18 Oct 2018 Last revised: 15 May 2019

Date Written: May 1, 2019


Different judges, doctors, loan officers, and patent examiners make different decisions, generating costly uncertainty over ultimate outcomes. In this paper, I use multiple-stage decisionmaking institutions to identify nonparametric bounds on disagreement between decision-makers. I bound disagreement to at least 17% of all Canadian refugee appeals, 150% larger than the estimate using existing methods and substantial relative to an average approval rate of 14%. I aggregate disagreement into judge-specific measures of quality, and find that quality improves with experience, declines with workload, and is higher for judges appointed under a nonpartisan regime. Finally, I adopt my method to test and reject the typical examiner-assignment monotonicity assumption.

Keywords: judge quality, judge fixed effect, examiner-assignment, IV monotonicity

JEL Classification: K1, K37, C26

Suggested Citation

Norris, Samuel, Examiner Inconsistency: Evidence from Refugee Appeals (May 1, 2019). University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2018-75. Available at SSRN: https://ssrn.com/abstract=3267611 or http://dx.doi.org/10.2139/ssrn.3267611

Samuel Norris (Contact Author)

University of Chicago ( email )

1155 East 60th Street
Chicago, IL 60637
United States

Register to save articles to
your library


Paper statistics

Abstract Views
PlumX Metrics