Credibility in Empirical Legal Analysis

56 Pages Posted: 8 Aug 2019 Last revised: 17 Mar 2022

See all articles by Hillel J. Bavli

Hillel J. Bavli

Southern Methodist University - Dedman School of Law

Date Written: August 7, 2019

Abstract

Empirical analysis is central in both legal scholarship and litigation, but it is not credible. Researchers can manipulate data to arrive at any conclusion they wish to obtain. A practice known as data fishing—searching for and selectively reporting methods and results that are favorable to the researcher—entirely invalidates a study’s results by giving rise to false positives and false impressions. Nevertheless, it is prevalent in law, leading to false claims, incorrect verdicts, and destructive policy. In this article, I examine the harm that data fishing in empirical legal research causes. I then build on methods in the sciences to develop a framework for eliminating data fishing and restoring confidence in empirical analysis in legal scholarship and litigation. This framework—which I call DASS (an acronym for Design, Analyze, Scrutinize, and Substantiate)—is designed to be simple, flexible, and practical for application in legal settings. It provides a concrete method for researchers to use to safeguard against data fishing and for consumers of empirical analysis to use to evaluate a researcher’s empirical claims. Finally, after describing the DASS framework and its application in various legal settings, I consider its implications for the “hired-gun” problem and other difficulties related to the reliability of expert evidence.

Keywords: empirical legal studies, data fishing, reliability, accuracy, hypothesis test, quantitative analysis, methodology, research practices, expert evidence, empirical analysis, hired gun, false positives, battle of the experts, statistical analysis, data manipulation, scientific evidence, p-value

Suggested Citation

Bavli, Hillel, Credibility in Empirical Legal Analysis (August 7, 2019). 87 Brooklyn Law Review 501 (2022), SMU Dedman School of Law Legal Studies Research Paper No. 434, Available at SSRN: https://ssrn.com/abstract=3434095 or http://dx.doi.org/10.2139/ssrn.3434095

Hillel Bavli (Contact Author)

Southern Methodist University - Dedman School of Law ( email )

P.O. Box 750116
Dallas, TX 75275
United States

HOME PAGE: http://bit.ly/Bavli-Hillel-J

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
203
Abstract Views
1,391
Rank
277,452
PlumX Metrics