A Framework for Evaluating Reproducibility and Replicability in Economics

22 Pages Posted: 25 May 2023 Last revised: 12 Jun 2023

See all articles by Anna Dreber

Anna Dreber

Stockholm School of Economics - Department of Economics

Magnus Johannesson

Stockholm School of Economics - Department of Economics

Date Written: May 24, 2023

Abstract

A fundamental question to the scientific enterprise is to what extent published scientific findings are credible. This question is related to the reproducibility and replicability of scientific findings where reproducibility is defined as testing if the results of an original study can be reproduced using the same data and replicability is defined as testing if the results of an original study hold in new data. We provide a framework for evaluating reproducibility and replicability in economics and divide reproducibility and replicability studies into five types: computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability and conceptual replicability, and we propose indicators to be reported for each type.

Keywords: reproducibility, replicability, computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability, conceptual replicability

JEL Classification: C18, C19

Suggested Citation

Dreber, Anna and Johanneson, Magnus, A Framework for Evaluating Reproducibility and Replicability in Economics (May 24, 2023). Available at SSRN: https://ssrn.com/abstract=4458153 or http://dx.doi.org/10.2139/ssrn.4458153

Anna Dreber

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden

Magnus Johanneson (Contact Author)

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden
+46 8 736 9443 (Phone)

HOME PAGE: http://www.hhs.se/Faculty/showperson.htm?personid=198

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
97
Abstract Views
316
Rank
461,862
PlumX Metrics