Science Deserves Better: The Imperative to Share Complete Replication Files

Abstract

Scientific knowledge is only as reliable as the empirical analysis on which it is based. For the majority of published statistical analyses, readers have to trust that the scholars correctly implemented the many stages of analysis between primary data collection and the presentation of results — including data cleaning, merging, recoding and transforming, analysis, and output. I advocate the adoption of a simple transparency maxim: good research involves publishing complete replication files, making every step of research as explicit and reproducible as is practical. Benefits of replication transparency include: raising the quality and refutability of scientific inferences; dissemination of useful data and code; greater freedom for scientists to explore each others’ results and data; and a richer scientific conversation. More transparent replication prac- tices are a scientific public good: the costs are small but borne by the authors; the benefits are great and shared by the broader scientific community and public. Strong norms of replication transparency, once established, are likely to be partly self-enforcing. Data is presented on replication practices and the replicability of published works in political science. I outline good replication practices for scholars, and offer recommendations to journals, universities, and funders.