Measurement Equivalence in Replications of Experiments: When and Why it Matters and Guidance on How to Determine Equivalence

Abstract

Replications of experiments are typically conducted to verify initial findings, increase their external validity, or to study the boundary conditions of treatment effects. A crucial and implicitly made assumption is that outcome measures in experiments are sufficiently comparable (i.e., equivalent) across experimental settings. We argue that there are good reasons to believe that this equivalence assumption may not always be met and should therefore be tested empirically. Integrating the literatures on experimental replication and survey measurement equivalence, we provide guidance when and how experimental replicators need to determine cross-replication equivalence.

Keywords: Replication, experiment, measurement equivalence, measurement invariance, cross-replication equivalence, blame, blame shifting, responsibility attribution, political accountability; measurement

JEL Classification: D12, D18, D4, D63, D8