The Replication Recipe: What Makes for a Convincing Replication?

Abstract

Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge base; however, there is not a consensus about what constitutes a convincing replication study. To facilitate close and convincing replication attempts we have developed a Replication Recipe, outlining standard criteria for a convincing close replication. This includes faithfully recreating the original study while keeping track of differences, achieving high statistical power, checking the study’s assumptions in new contexts, and pre-registering the study. We also discuss methods for evaluating and reporting replications. Identifying differences between replication and original (sample, culture, lab context, etc.) allows researchers to identify where their replication is on the continuum from “close” to “conceptual”. Our replication recipe can be used by established researchers, teachers, and students to conduct meaningful replication studies and integrate replications into their scholarly habits.

Keywords: replications, replication recipe, statistical power, research methods, pre-registration, solid science