A Framework for Evaluating Reproducibility and Replicability in Economics

24 Pages Posted: 25 May 2023 Last revised: 30 Nov 2023

See all articles by Anna Dreber

Anna Dreber

Stockholm School of Economics - Department of Economics

Magnus Johannesson

Stockholm School of Economics - Department of Economics

Date Written: May 24, 2023

Abstract

We propose a framework for evaluating reproducibility and replicability in economics. Reproducibility is defined as testing if the results of an original study can be reproduced using the same data and replicability is defined as testing if the results of an original study hold in new data. We further divide reproducibility and replicability studies into five types: computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability and conceptual replicability. In addition to this typology we propose indicators to measure the degree of reproducibility and replicability in both individual studies and for a group of studies.

Keywords: reproducibility, replicability, computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability, conceptual replicability

JEL Classification: C18, C19

Suggested Citation

Dreber, Anna and Johanneson, Magnus, A Framework for Evaluating Reproducibility and Replicability in Economics (May 24, 2023). Available at SSRN: https://ssrn.com/abstract=4458153 or http://dx.doi.org/10.2139/ssrn.4458153

Anna Dreber

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden

Magnus Johanneson (Contact Author)

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden
+46 8 736 9443 (Phone)

HOME PAGE: http://www.hhs.se/Faculty/showperson.htm?personid=198

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
170
Abstract Views
494
Rank
311,837
PlumX Metrics