Replication and the Manufacture of Scientific Inferences: A Formal Approach
32 Pages Posted: 17 Sep 2014 Last revised: 29 May 2015
Date Written: May 28, 2015
The field of replication studies remains a controversial, misunderstood, and unappreciated piñata of 18 replication typologies, spanning 79 replication types. To help bring order into chaos I contribute a theory of manufactured inferences. The theory is built on three pillars: (i) replication causal diagrams (or r-dags, for short); (ii) a formal conceptualization of study procedures; and (iii) the use of Bayesian inference to update our beliefs about the natural phenomenon under investigation, and the operating characteristics of the study procedures used to study it. Next, I use this theory to motivate a formal typology of replications types; explaining how they are done, and for what purpose. Finally, I discuss some implications of this theory, including the importance of an analytical approach to robustness and generalizability replications; the need to avoid conceptual replications; the possibility of legitimate (unplanned) specifications searches; the limitations of meta-analysis; and the false dichotomy between “successful” and “failed” replications.
Keywords: Replication, replication types, robustness, experimental artifacts, replica- tion causal diagrams, quality control, active learning
JEL Classification: C11, C14, C44, C52, C90, C93
Suggested Citation: Suggested Citation