Law and Psychology Grows Up, Goes Online, and Replicates

36 Pages Posted: 7 May 2018

See all articles by Krin Irvine

Krin Irvine

University of Chicago - Booth School of Business

David A. Hoffman

University of Pennsylvania Law School; Cultural Cognition Project at Yale Law School

Tess Wilkinson‐Ryan

University of Pennsylvania

Multiple version iconThere are 2 versions of this paper

Date Written: June 2018

Abstract

Over the last 30 years, legal scholars have increasingly deployed experimental studies, particularly hypothetical scenarios, to test intuitions about legal reasoning and behavior. That movement has accelerated in the last decade, facilitated in large part by cheap and convenient Internet participant recruiting platforms like Amazon Mechanical Turk. The widespread use of online subjects, a practice that dramatically lowers the barriers to entry for experimental research, has been controversial. At the same time, the field of experimental psychology is experiencing a public crisis of confidence widely discussed in terms of the “replication crisis.” At present, law and psychology research is arguably in a new era, in which it is both an accepted feature of the legal landscape and also a target of fresh skepticism. The moment is ripe for taking stock. In this article, we bring an empirical approach to these problems. Using three canonical law and psychology findings, we document the challenges and the feasibility of reproducing results across platforms. We evaluate the extent to which we are able to reproduce the original findings with contemporary subject pools (Amazon Mechanical Turk, other national online platforms, and in‐person labs). We partially replicate all three results, and show marked similarities in subject responses across platforms. In the context of the experiments here, we conclude that meaningful replication requires active intervention in order to keep the materials relevant and sensible. The second aim is to compare Amazon Mechanical Turk subjects to the original samples and to the replication samples. We find, consistent with the weight of recent evidence, that the Amazon Mechanical Turk samples are reasonably appropriate for these kinds of scenario studies. Subjects are highly similar to subjects on other online platforms and in‐person samples, though they differ in their high level of attentiveness. Finally, we review the growing replication literature across disciplines, as well as our firsthand experience, to propose a set of standard practices for the publication of results in law and psychology.

Suggested Citation

Irvine, Krin and Hoffman, David A. and Wilkinson‐Ryan, Tess, Law and Psychology Grows Up, Goes Online, and Replicates (June 2018). Journal of Empirical Legal Studies, Vol. 15, Issue 2, pp. 320-355, 2018. Available at SSRN: https://ssrn.com/abstract=3173374 or http://dx.doi.org/10.1111/jels.12180

Krin Irvine (Contact Author)

University of Chicago - Booth School of Business

5807 S. Woodlawn Avenue
Chicago, IL 60637
United States

David A. Hoffman

University of Pennsylvania Law School ( email )

3501 Sansom Street
Philadelphia, PA 19104
United States

Cultural Cognition Project at Yale Law School

127 Wall St
New Haven, CT 06520
United States

Tess Wilkinson‐Ryan

University of Pennsylvania

Philadelphia, PA 19104
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
1
Abstract Views
191
PlumX Metrics