What Irreproducible Results Mean for the Law of Scientific Evidence

Abstract

In 2015 Brian Nosek and several collaborators performed perhaps the most important scientific study of the year, and they did it by attempting to copy the work of others. That is not as contradictory as it sounds, or at least it shouldn’t be.

While the conventional wisdom is that most scientific findings have been vetted and reproduced many times before they reach scientific consensus, the reality is that such rigor is exceedingly rare. Nosek sought to remedy this, wrangling up a group of 270 other researchers (The Open Science Collaboration) and attempting to redo 100 psychology experiments already published in leading peer reviewed journals. The question: would these recreations find the same results as the initial studies? Prior to Nosek’s endeavor, the conventional wisdom was that while published science contains some false positives, they are a small minority and are quickly identified through a robust self-correction process. The conventional wisdom was wrong, and to an extent most never dreamed of – only 36% of the studies were reproducible.

In this article I first provide a brief review of the legal standard for the admission of scientific evidence. Then, I provide the context in which Nosek’s findings sit – the replicability crisis. I suggest the current legal standard is capable of effectively screening out unreliable science, but it requires legal actors be aware of the issue and demand science’s best rather that what’s been generally accepted. I conclude with some tangible recommendations for the justice system.

Keywords: Evidence, psychology and law, replicability crisis, science and law, scientific evidence

JEL Classification: K10, K14, K41