A Plea for More Aggregation: The Looming Threat to Empirical Legal Scholarship

Abstract:     
Empirical scholarship in the social sciences and the law stands at a critical threshold. As the volume of statistical analysis grows, it becomes increasingly difficult to assess what we actually know about a particular phenomenon. Contradictory findings abound, and the social sciences and empirical legal studies currently lack effective tools to filter the good claims from the bad and to synthesize the high-quality findings. This paper explores how to fundamentally change and improve the way these fields generate and use empirical knowledge.

I start by examining why these fields are poorly equipped to draw conclusions from an empirical literature. The root problem is a misguided philosophy of science. Contrary to what social scientists are taught and believe, empirical knowledge does not come from (deductively) testing hypotheses but rather from (inductively) measuring effect sizes. Induction, however, requires a far more holistic perspective than deduction. Given their mistaken philosophical foundation, the social sciences and empirical legal scholarship have undervalued such perspectives.

I then argue that the solution is to develop rigorous, evidence-based systematic reviews for observational research. These reviews have been revolutionizing fields like medicine and epidemiology that use randomized clinical trials, but they are almost unheard of in areas that rely on observational work. I demonstrate why these reviews are substantially superior to the more informal approaches currently used. I also discuss many of the significant challenges that face those who wish to design such reviews for observational work and point to possible solutions.

Number of Pages in PDF File: 52

Keywords: Econometrics, Hypothesis Testing, Evidence-Based Policy, Quality Guidelines

JEL Classification: C10, C12, K19

working papers series