A Practical Guide to Registered Reports for Economists

Abstract

The current publication system in economics has encouraged the inflation of positive results in empirical papers. Registered Reports, also called Pre-Results Reviews, are a new submission format for empirical work that take pre-registration one step further. In Registered Reports, researchers write their paper before running the study and commit to a detailed data collec- tion process and analysis plan. After a first-stage review, a journal can give an In-Principle- Acceptance guaranteeing that the paper will be published if the authors carry out their data collection and analysis as pre-specified. We here propose a practical guide to Registered Reports for empirical economists. We illustrate the major problems that Registered Reports address (p- hacking, HARKing, forking, and publication bias), and present practical guidelines on how to write and review Registered Reports (e.g., the data-analysis plan, power analysis, and correction for multiple-hypothesis testing), with R and STATA codes. We provide specific examples for experimental economics, and show how research design can be improved to maximize statisti- cal power. Last, we discuss some tools that authors, editors and referees can use to evaluate Registered Reports (checklist, study-design table, and quality assessment).

Keywords: registered reports, practical guide, pre-registration, p-hacking, HARKing, multiple-hypothesis testing, power analysis, smallest effect size of interest

JEL Classification: A10, C12, C9