Scientific Progress in the Absence of New Data: A Procedural Replication of Ross (2006)
38 Pages Posted: 25 Apr 2013 Last revised: 22 Jan 2014
Date Written: November 11, 2013
Estimating the effect of macro variables like democracy on aggregate outcomes like child mortality remains a formidable challenge. When new data are limited, theories imprecise, and experimentation impossible, findings are mostly driven by modeling assumptions and inadvertent errors. Here I propose procedural replication as a method of generating objective evidence capable of demonstrating new insights even in the absence of new data. I develop a simple Bayesian framework for replication studies, distinguish five different types of replication, and show how procedural replication can improve answers to existing research questions, tether inferences to data, and generate checklists for cumulative research. A procedural replication of Ross's (2006) controversial finding that democracy has no effect on child mortality shows this null finding to be an artifact of the way quinquennial averages were computed, and the static nature of the preferred model. I address other shortcomings and provide a procedural checklist to inform future studies.
Keywords: procedural replication, experimental artifact, procedural errors, checklists, scientific standards, causal inference, observational studies, democracy, infant mortality
JEL Classification: B4, C1, C80, I18, P00, P16, P26
Suggested Citation: Suggested Citation