When Forensic Examiners Disagree: Bias, or Just Inaccuracy?

Abstract:     
Background: Previous investigators have suggested that bias might account for the disparate rates at which examiners conclude that defendants are competent to stand trial (CST). This article describes three computer studies of how biases and imperfect accuracy might affect rates of disagreement.

Methods: Study 1 assumed that examiners could discriminate between competent and incompetent accurately (effect size = 1.81, ROC area = 0.90) and used computer simulation of 20,000 pairs of CST evaluations to determine how different judgment thresholds (e.g., thresholds exemplifying biases toward opinions that defendants were competent or incompetent) would elevate disagreement rates above those expected through chance error alone. Studies 2 and 3 evaluated the assumptions of Study 1 using previously published data to make inferences about examiner accuracy and threshold locations.

Results: Imperfect accuracy alone would often explain much between-examiner disagreement, even if examiners approached evaluations with distinct biases. Results from Studies 2 and 3 suggested that assumptions used in Study 1 were reasonable.

Conclusion: Many instances of between-examiner disagree might be attributable to imperfect accuracy that expresses itself in random errors, rather than to examiner biases that imply different thresholds for judging defendants’ competence.

Number of Pages in PDF File: 35

Keywords: competence to stand trial, receiver operating characteristic, ROC, gold standard, diagnostic accuracy, reliability, forensic mental health, expert opinion, bias

JEL Classification: C11, C14, C15, I00, K14

Accepted Paper Series