Signal Detection Theory Fails to Account for Real-World Consequences of Inconclusive Decisions

Law, Probability and Risk (2022) 21, 131–135

U of Maryland Legal Studies Research Paper No. 2023-02

6 Pages Posted: 24 Apr 2023

See all articles by Maneka Sinha

Maneka Sinha

University of Maryland Francis King Carey School of Law

Richard E. Gutierrez

Office of the Cook County Public Defender

Date Written: February 7, 2023

Abstract

This response to Arkes and Koehler’s important paper, Inconclusives and Error Rates In Forensic Science: A Signal Detection Theory Approach, explains that the authors’ approach: (1) neglects to account for known differences in how inconclusive decisions are deployed by examiners across various feature-comparison methods and (2) leaves readers with the misimpression that blind proficiency testing may serve as a near-complete solution to the complex problems associated with assessing the validity of feature-comparison methods.

Arkes and Koehler describe three categories of feature-comparison examiners: conservative (focused on minimizing error); confident (focused on drawing a firm source conclusion); and strategic (acting conservatively or confidently depending on whether they are in a test situation or examining case work). While it may apply to some disciplines under certain scenarios, firearms examiners routinely apply an approach that falls outside of this scheme. The published literature demonstrates that firearms examiners do not draw inconclusive decisions evenly between same-source and different-source comparisons. Rather, they consistently render inconclusive decisions more frequently when data supports exclusion rather than identification.

This approach is best described by a fourth category, which we term, the Biased Examiner. As this response explains, the Biased Examiner problem results in significant prejudice to the accused and may contribute to miscarriages of justice. And, because it stems from a number of sources, the Biased Examiner problem cannot be fully resolved by blind proficiency testing.

Keywords: forensic evidence, friction ridge, firearms examiners, wrongful convictions, comparison algorithms, systemic bias

Suggested Citation

Sinha, Maneka and Gutierrez, Richard E., Signal Detection Theory Fails to Account for Real-World Consequences of Inconclusive Decisions (February 7, 2023). Law, Probability and Risk (2022) 21, 131–135 , U of Maryland Legal Studies Research Paper No. 2023-02, Available at SSRN: https://ssrn.com/abstract=4362641

Maneka Sinha (Contact Author)

University of Maryland Francis King Carey School of Law ( email )

500 West Baltimore Street
Baltimore, MD 21201-1786
United States

Richard E. Gutierrez

Office of the Cook County Public Defender

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
52
Abstract Views
303
Rank
691,921
PlumX Metrics