• Selected
• Entire List

Abstract:
That would make the error rate [in felony convictions] .027 percent - or to put it another way, a success rate of 99.973 percent.

- Justice Antonin Scalia, concurring in Kansas v. Marsh, June 29, 2006 (quoting Joshua Marquis)

The news about the astounding accuracy of felony convictions in the United States, delivered by Justice Scalia and Joshua Marquis in the passage set out epigrammatically above, would be cause for rejoicing if it were true. Imagine. Only 27 factually wrong felony convictions out of every 100,000! Unfortunately, it is not true, as the empirical data analyzed in this article demonstrates.

To a great extent, those who believe that our criminal justice system rarely convicts the factually innocent and those who believe such miscarriages are rife have generally talked past each other for want of any empirically-justified factual innocence wrongful conviction rate. This article remedies at least a part of this problem by establishing the first such empirically justified wrongful conviction rate ever for a significant universe of real world serious crimes: capital rape-murders in the 1980's.

Using DNA exonerations for capital rape-murders from 1982 through 1989 as a numerator, and a 406-member sample of the 2235 capital sentences imposed during this period, this article shows that 21.45%, or around 479 of those, were cases of capital rape murder. Data supplied by the Innocence Project of Cardozo Law School and newly developed for this article show that only 67% of those cases would be expected to yield usable DNA for analysis. Combining these figures and dividing the numerator by the resulting denominator, a minimum factually wrongful conviction rate for capital rape-murder in the 1980's emerges: 3.3%.

The article goes on to consider the likely ceiling accompanying this 3.3% floor, arriving at a slightly softer number for the maximum factual error rate of around 5%. The article then goes on to analyze the implications of a factual error rate of 3.3%-5% for both those who currently claim errors are extremely rare, and those who claim they are extremely common. Extension of the 3.3%-5% to other capital and non-capital categories of crime is discussed, and standards of moral duty to support system reform in the light of such error rates is considered at length.

Criminal Law, Criminal Procedure, Wrongful Convictions, Evidence, Proof, Standard of Proof, Empirical Wrongful Conviction Rate, Actual Innocence, Unsafe Verdict

Abstract:
After the Supreme Court's decision in Kumho Tire v. Carmichael and the recent amendment of Federal Rule of Evidence 702, proffers of expert testimony will have to be found reliable for the particular application of the asserted expertise to the "task at hand." That is, expertise which is reliable in some global sense, which might apply to other cases but not to the particular application before the court, does not satisfy the requirements for admission. With that in mind, this article examines the phenomoenon of "observer effects" and the vulnerability of forensic science examinations to such observer effects. Observer effects occur when the results of an examination are distorted by the context and state of the observer, including the observer's expectations and desires. The article reviews the findings and practices of a range of scientific fields concerning such observer effects and their control, with special attention to the relevant research and theory from cognative and social psychology. This literature establishes that in virtually every area of human judgment, such observer effects have a relentless and sometimes dramatic effect on the accuracy of results. The article then examines current forensic science practice in light of that research, concluding that forensic science practice is far behind most scientific fields in controlling for such effects, leaving the reliability and accuracy of many forensic science results in doubt. The article then suggests practical ways in which forensic science practice can be changed to reduce such problems, such as the adoption of blind testing regimes. Finally, the article analyzes the current state of the law under Kumho Tire and Rule 702, concluding that the results of forensic science examinations are in danger of being excluded if their reliability continues to be undermined by the failure to control observer effects.

Evidence, experts, forensic science, observer effects, context effects, expectation, suggestion, Kumho

Abstract:
The NAS Committee Report, STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES, issued in February of 2009, was a milestone in the decades-long struggle to get those who control the production and utilization of forensic science expertise to admit the various weaknesses of some of the techniques involved, and to take steps to strengthen the reliability of those techniques and their products. The NAS Committee Report is in some ways the culmination of those efforts, and has made it now untenable to dismiss criticisms as simply the cavils of uninformed academics with nothing better to do.

In this sense the report is a glass nine-tenths full, and is to be celebrated as such. But then there is the other tenth, the tenth that may, as an unintended consequence, delay needed reform significantly and unnecessarily. The most significant part of this unwise tenth is the decision not to push strongly for the immediate adoption of masking and sequential unmasking protocols in forensic science practice, but instead to call for “more research” on the issue in advance of moving forward.

This paper explains in detail why the “await more research” approach is misguided.

evidence, criminal practice, criminal procedure, forensic science, expert witnesses, expertise, blind testing, sequential unmasking, NAS Committee report

Abstract:
In a recent article in the Texas Tech Law Review entitled Deadly Dilemmas, Ronald Allen and Larry Laudan make a number of claims about the performance of the criminal justice system. On an empirical level, they assert that by using data developed in earlier articles by me and by Brandon Garrett, they can establish that (taking pleas into account) fewer than 1% of those convicted of 'violent crimes' are factually innocent. On a broader normative level, they claim that many observers of the criminal justice system worry too much about the problem of inaccurate convictions, and not enough about the problem of inaccurate acquittals. They further assert that, based on the relative risks to the average individual of being a crime victim vs. being wrongfully convicted of a crime, 'the social contract' requires that more attention be paid to reforms reducing the number of acquittals of the guilty, including potentially adjustments to the criminal standard of proof. This article examines (and disputes) those claims in detail.

Abstract:
This piece seeks to collect and separately describe and analyze every explicit decision by an American court on the reliability of handwriting identification expertise from the decision in Daubert in 1993 to March of 2008. It serves as an appendix to my article Goodbye to All That, or, A Fool's Errand, By One of the Fools: How I Stopped Worrying About Court Responses to Handwriting Identification (and Forensic Science in General) and Learned to Love Misinterpretations of Kumho Tire v. Carmichael (separately published at 43 Tulsa L. Rev. 477). As to each of the 68 cases treated, I have identified the judges and experts involved, and given a detailed analysis both of the facts of the case and the quality of the factual and legal analysis involved in the court's opinion.

Some explanations of methodology and criteria of inclusion are in order. In compiling this piece, I have excluded all cases where there was no reliability challenge of any sort, and all cases where document examiners were rejected merely because they failed to possess sufficient training or experience according to whatever standards that particular court applied that day, but would have, by implication, allowed a better credentialed expert to testify. (There is, of course, no official standard for that criterion in any jurisdiction, so far as I know.) It should also be understood that the cases listed here are by no means all of the cases since 1993 in which handwriting identification testimony by putative experts has been proffered or accepted. A search of the Westlaw Allcases database using an appropriate search string will reveal a couple of thousand cases where claimed handwriting identification expertise has played a role and no reliability challenge has been made, and that is just cases which generated opinions that showed up on Westlaw. Most use of such expertise likely goes unremarked upon, or occurs in cases that never generate written opinions. In the vast majority of the reported cases involving such experts, the testimony is merely noted as part of a recitation of facts. These cases include substantial numbers of civil cases, often involving challenged signatures on wills or deeds, or insurance and other contract cases, but not uncommonly involving more complex issues. The volume should not be surprising. Estimates of the number of persons who offer such testimony in court, at least on occasion, ranges up to 5000 or more, with some hundreds who do so regularly. The range of credentials and experience exhibited by these witnesses is also startling, and it is likely that most of the testimony that occurs in American courtrooms is by persons whose training and experience would be looked down upon by the establishment accrediting body, the American Board of Forensic Document Examiners (ABFDE).

Because the reasons to believe that various applications of claimed handwriting identification expertise are sufficiently reliable for the purposes of the criminal law (or not) depends in large part on the amount and quality of empirical research that has or has not been done on those issues, and because the evaluation of judicial performance can only be done in light of that research and the courts' treatment of it (including whether it is dealt with in the opinion under discussion or not), it is necessary to the purpose of this piece to set out a summary of the currently available research on these matters, which is done in Part II, before the individual cases are analyzed.

Evidence, criminal evidence, expert evidence, scientific evidence, Daubert, Kumho Tire, legal process

Abstract:
Federal Rule of Evidence 401 appears on its face to adopt a “god-view” standard of relevance, which looks only to the relationship between a proffer (the probans) and a target issue (the probandum) in some cosmic sense. This approach is deeply flawed, and renders the resulting notion of “logical relevance” both illogical and unable to do any work whatsoever. Instead, any workable concept of relevance must be seen as resulting from a tri-partite relationship between and among the probans, the probandum and the processor, that is, the human who acts as the “decoder” of the information (factfinder).

This approach renders “relevance” as much a function of the decoder as the code. It makes the program of applying “naturalized epistemology” to the legal system not merely desirable, but necessary. And it carries significant implications for how to approach the question of evidence reform.

evidence, relevance, inference, naturalized epistemology

Abstract:
This article is a picaresque romp through the author's career, much of which has been spent coming to grips with the realities of forensic science, and the courts' abdication of their role as gatekeepers in judging the reliability of prosecution-proffered expertise. The reader follows the author from Bruno Hauptmann's cramped attic, through the Mayflower Madam case and into the era of Daubert and beyond. There is a serious academic point to all this, for the article illustrates how the lower federal courts have managed to ignore or misinterpret Kumho Tire v. Carmichael in such a way as to create a jurisprudence of expertise wholly at odds with the clear mandate of the Supreme Court, often by converting decisions with no precedential status into precedents of breathtaking breadth. In this regard, the article is as much about legal process in general as it is about expert evidence in criminal cases.

Evidence, criminal evidence, expert evidence, scientific evidence, Daubert, Kumho Tire, legal process

Abstract:
It is an asserted principle of ancient lineage dating at least to the 17th century that, while standards of proof may vary depending on the kind of case or issue involved, rules of admissibility (at least those not explicitly based on policies extrinsic to rectitude of decision) ought ideally to be the same in every context, civil or criminal. It was an explicit policy of Wigmore to bring the rules of evidence more in line with this ideal. The reason given has always been that information tending to prove a fact has that same tendency whether the fact is a material issue in a criminal case or in a civil case, for the plaintiff, prosecution or defense.

If all we asked the factfinder to do in our system of litigation was to deal with fact reconstruction strictly speaking, and if contextual pressures on factfinders attempting to evaluate evidence were the same in every context, there would be some force to this argument. But it is clear that our trial system assigns other important functions to the jury besides fact reconstruction strictly defined, and that the pressures and potential distortions of contextual atmosphere in which we expect factfinders to work varies greatly from case to case.

To be sure, we ask factfinders to determine the empirical details of facts in the world. But we also ask factfinders to make some determinations which are clearly not facts (e.g. negligence; insanity), but are rather value judgments about facts. And we ask factfinders to determine other things (states of mind) whose factual status is epistemically different from (and less clear than) exterior facts in the world, and which are generally inseparable from normative evaluations of responsibility and guilt. Might the kinds of information which are appropriate for one function be inappropriate to the proper performance of other functions? In addition, in the criminal setting, might the mindset which is directed toward investigating and producing evidence of guiltiness not be that best suited to investigating and producing reliable evidence of guilt? Can the proper structuring of the trial, and adjustments in the rules of evidence, help resolve this and similar conflicts, or by the time the trial starts is it already too late, given the potentials for the pre-trial distortion of information which are inevitably a part of a system run largely by partisan adversaries?

Evidence, trial, Evidence, criminal evidence, expert evidence, scientific evidence, adversary system, legal process

Abstract:
This article shows that, as to profferes of asserted expert testimony, civil defendants win their Daubert reliability challenges to plaintiff's profferes most of the time, and that criminal defendant's virtually always lose their reliability challenges to government proffers. And, when civil defendants' profferes are challenged by plaintiffs, those defendants uisually win, but when criminal defendants' proffers are challenged by the prosecution, the criminal defendants usually lose. The article then goes on to examine, in detail, various categories of expert proffers in criminal cases, including "syndrome evidence", polygraph, bite mark, handwriting, modus operandi, and eye witness weakness, to shed light on whether the system bias revealed in the statistical breakdown is illusory or real. Finally, an afterword analyzes the last year's cases, and makes observations on apparent trends.

Abstract:
DNA analysis has resulted in a troubling number of exonerations in both capital and non-capital cases. While these cases show that significant numbers of factually innocent persons are convicted of crimes that they did not commit, most such convictions remain hidden because they occur in cases where DNA analysis has no application. This article seeks to show the coordinate failure of the two main current visions of the trial. On one hand, the standard model of the trial has obscured the proper normative warrant of the jury, while at the same time inappropriately insulating jury verdicts of guilt from review because of excessive deference to jury evaluation of live testimony. On the other hand, the model of the trial put forth by adversary enthusiasts celebrates the jury's normative warrants, but obscures the shortcomings of current adversary processes when such a normative warrant is inapplicable, that is, in criminal cases where the practical issue is the actual innocence in fact of the defendant. Either account of the trial allows judges, especially appellate judges, to avoid responsibility for conviction of the factually innocent. This article asserts that claims of actual innocence in fact (strictly defined) possess a moral purchase far superior to other moral claims that animate the legal process. It proposes reforms intended to recognize the special moral position of innocence-in-fact claims and to make real the legal system's commitment to truly responsive standards of reasonable doubt in regard to such claims. Specifically, the article proposes special trial rules for such claims aimed at curbing adversary excess, and review of convictions in such cases by a new standard of review borrowed in part from British jurisprudence, the "unsafe verdict" standard.

Abstract:
This article asserts that Kumho Tire v. Carmichael, 526 U.S. 137 (1999) stands for two important priciples: first, that the gatekeeping requirement of minimum threshold reliability pursuant to Federal Rule of Evidence 702 applies to all profferred expert testimony, not just to the explicit products of "science"; and second, that this threshold reliability must be evaluated in regard to the particular "task at hand", not globally in regard to the average dependability of a broadly defined area of expertise, which might be dependable in other contexts, but not in regard to the "task at hand". The first of these principles is explicit in the Kumho Tire opinion, and the second somewhat less obvious, but of central importance and potentially of even greater impact. The article explains in detail why the decision must be read as standing for the second principle, relying both on the text of the opinion and the notions of reliability which it embodies. It then goes on to examine what the effect of appropriate task-at-hand analysis would be on forensic science admissibility issues in criminal cases, illustrating its impact through an analysis of all the opinions on the admissibility of handwriting identification testimony since the decision of Daubert v. Merrell Dow Pharmaceuticals, Inc. 509 U.S. 579 (1993).

Kumho, Daubert, evidence, expert, criminal evidence, handwriting identification

Abstract:
If modern cognitive psychology teaches us anything, it is that humans are vulnerable to some predictable kinds of processing errors. We appear to be subject to a variety of cognitive tunnels from which it is difficult to escape even when they lead to error. Many of those cognitive tunnels deal with probability judgments. This is well illustrated by the famous Monte Hall problem, and by the results of various studies by Daniel Kahneman, Amos Tverski and others. Some, most notably Gerd Gigerenzer, have responded that these results do not show a fundamental irrationality, but are merely the side effects of “fast and frugal” heuristics developed by evolutionary pressures to allow quick and generally accurate processing in informationally rich environments. However, modern social arrangements may make heuristics that were beneficial when they evolved generate seriously wrong decisions in the informational environment in which we now operate. For instance, credibility judgments concerning other humans that worked well in small bands of acquaintances may not work so well with strangers in the big city, as anyone falling prey to a Ponzi scheme could attest. As our information derives from less and less local environments, our hard wired heuristics become less and less secure guides. A simple mind experiment involving multiple roulette wheels spun simultaneously in two separate rooms, establishes that in any informationally rich environment, there are coincidences that appear to be rare and meaningful, but are neither.
One form of supposed expertise sometimes offered in criminal proceedings, and more often used in criminal investigation, “linkage analysis,” presents just such dangers. This asserted expertise, and the larger area from which it is derived, “offender profiling,” are rife with claims for which little empirical evidence exists. The history and empirical record relating to “linkage analysis” and “offender profiling” is examined, and the dangers they present illustrated by reference to various actual cases, particularly the New Jersey case of State v. Fortin.

Evidence, Criminal Law, Criminal Procedure, Forensic Science, Cognitive Psychology