Definition and Diagnosis of Problematic Attrition in Randomized Controlled Experiments
Fernando Martel García
Cambridge Social Science Decision Lab Inc.
May 26, 2013
Attrition is the Achilles' Heel of the randomized experiment: It is fairly common, and it can completely unravel the benefits of randomization. Using the structural language of causal diagrams I demonstrate that attrition is problematic for identification of the average treatment effect (ATE) if -- and only if -- it is a common effect of the treatment and the outcome (or a cause of the outcome other than the treatment). I also demonstrate that whether the ATE is identified and estimable for all units in the experiment, or only for those units with observed outcomes, depends on two d-separation conditions. One of these is testable ex-post under standard experimental assumptions. The other is testable ex-ante so long as adequate measurement protocols are adopted. Missing at Random (MAR) assumptions are neither necessary nor sufficient for identification of the ATE.
Number of Pages in PDF File: 53
Keywords: attrition, randomized controlled experiments, field experiments, causal diagrams, directed acyclic graphs, average treatment effect, nonparametric
Date posted: July 30, 2013