A Framework for Evaluating Reproducibility and Replicability in Economics
22 Pages Posted: 25 May 2023 Last revised: 12 Jun 2023
Date Written: May 24, 2023
Abstract
A fundamental question to the scientific enterprise is to what extent published scientific findings are credible. This question is related to the reproducibility and replicability of scientific findings where reproducibility is defined as testing if the results of an original study can be reproduced using the same data and replicability is defined as testing if the results of an original study hold in new data. We provide a framework for evaluating reproducibility and replicability in economics and divide reproducibility and replicability studies into five types: computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability and conceptual replicability, and we propose indicators to be reported for each type.
Keywords: reproducibility, replicability, computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability, conceptual replicability
JEL Classification: C18, C19
Suggested Citation: Suggested Citation