Examiner Inconsistency: Evidence from Refugee Appeals
90 Pages Posted: 18 Oct 2018 Last revised: 15 May 2019
Date Written: May 1, 2019
Different judges, doctors, loan officers, and patent examiners make different decisions, generating costly uncertainty over ultimate outcomes. In this paper, I use multiple-stage decisionmaking institutions to identify nonparametric bounds on disagreement between decision-makers. I bound disagreement to at least 17% of all Canadian refugee appeals, 150% larger than the estimate using existing methods and substantial relative to an average approval rate of 14%. I aggregate disagreement into judge-specific measures of quality, and find that quality improves with experience, declines with workload, and is higher for judges appointed under a nonpartisan regime. Finally, I adopt my method to test and reject the typical examiner-assignment monotonicity assumption.
Keywords: judge quality, judge fixed effect, examiner-assignment, IV monotonicity
JEL Classification: K1, K37, C26
Suggested Citation: Suggested Citation