Quantifying the Accuracy of Forensic Examiners in the Absence of a Diagnostic 'Gold Standard'

Law of Human Behavior, Vol. 34, pp. 402-417, 2010

U of Cincinnati Public Law Research Paper No. 08-23

16 Pages Posted: 5 Oct 2008 Last revised: 17 Nov 2011

Douglas Mossman

University of Cincinnati College of Medicine

Michael D. Bowen

affiliation not provided to SSRN

David J. Vanness

affiliation not provided to SSRN

David Bienenfeld

affiliation not provided to SSRN

Terry Correll

affiliation not provided to SSRN

Jerald Kay

affiliation not provided to SSRN

William M. Klykylo

affiliation not provided to SSRN

Douglas S. Lehrer

affiliation not provided to SSRN

Date Written: November 16, 2011

Abstract

Background: In medicine, diagnostic accuracy is usually evaluated against a near-infallible criterion - a "gold standard" - for true disease status. Most mental health classifications have no gold standard, however, and absence of agreed-upon truth is common for psycholegal assessments.

Aims: To show that even without a gold standard, accuracy of forensic assessments can be quantified using latent class methods and receiver operating characteristic (ROC) analysis.

Method: Using redacted court reports, clinicians rated 156 hospitalized criminal defendants on their Dusky-defined competence to stand trial and on their understanding, appreciation, and reasoning about criminal proceedings. Multiple ratings per evaluee permitted estimation of ROC parameters using maximum likelihood and Bayesian approaches.

Results: Raters were highly accurate, with ROC areas averaging 0.967 or more.

Conclusions: Accuracy of forensic examinations can be quantified numerically despite the absence of a diagnostic gold standard. Methods used in this study should be applicable to many other psycholegal questions where quantifying accuracy would have scientific and evidentiary value.

Keywords: competence to stand trial, adjudicative competence, ROC analysis, diagnostic accuracy, maximum likelihood, Bayesian, gold standard

JEL Classification: C11, C14, C15, C63, K14

Suggested Citation

Mossman, Douglas and Bowen, Michael D. and Vanness, David J. and Bienenfeld, David and Correll, Terry and Kay, Jerald and Klykylo, William M. and Lehrer, Douglas S., Quantifying the Accuracy of Forensic Examiners in the Absence of a Diagnostic 'Gold Standard' (November 16, 2011). Law of Human Behavior, Vol. 34, pp. 402-417, 2010; U of Cincinnati Public Law Research Paper No. 08-23. Available at SSRN: https://ssrn.com/abstract=1278572 or http://dx.doi.org/10.2139/ssrn.1278572

Douglas Mossman (Contact Author)

University of Cincinnati College of Medicine ( email )

260 Stetson Street, Suite 3200
P. O. Box 670559
Cincinnati, OH 45219
United States
513-558-4423 (Phone)

Michael D. Bowen

affiliation not provided to SSRN

David J. Vanness

affiliation not provided to SSRN

David Bienenfeld

affiliation not provided to SSRN ( email )

Terry Correll

affiliation not provided to SSRN ( email )

Jerald Kay

affiliation not provided to SSRN ( email )

William M. Klykylo

affiliation not provided to SSRN ( email )

Douglas S. Lehrer

affiliation not provided to SSRN ( email )

Paper statistics

Downloads
110
Rank
204,327
Abstract Views
1,389