This letter comments on the report “Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-comparison Methods” released in September 2016, by the President’s Council of Advisors on Science and Technology (PCAST). The report advocates a two-stage procedure for evaluation of forensic evidence. The first stage is a “match”/“non-match” determination, and the second stage is an empirical assessment of sensitivity (correct acceptance) and false alarm (false acceptance) rates. Almost always, quantitative data from feature-comparison methods are continuously-valued and have within-source variability. We explain why a two-stage procedure is not appropriate for this type of data, and recommend more appropriate statistical procedures.
Morrison, Geoffrey Stewart and Morrison, Geoffrey Stewart and Kaye, David H. and Balding, David J. and Taylor, Duncan and Dawid, Alexander and Aitken, Colin and Gittelson, Simone and Zadora, Grzegorz and Robertson, Bernard W. and Willis, Sheila and Pope, Susan and Neil, Martin and Martire, Kristy A and Hepler, Amanda B. and Gill, Richard and Jamieson, Allan and de Zoete, Jacob and Ostrum, Brent and Caliebe, Amke, A Comment on the PCAST Report: Skip the 'Match'/'Non-Match' Stage (October 25, 2016). Forensic Science International (Forthcoming), Available at SSRN: https://ssrn.com/abstract=2860440