Definition and Diagnosis of Problematic Attrition in Randomized Controlled Experiments
50 Pages Posted: 25 Apr 2013
There are 3 versions of this paper
Definition and Diagnosis of Problematic Attrition in Randomized Controlled Experiments
When and Why is Attrition a Problem in Randomized Controlled Experiments and How to Diagnose it
Date Written: April 25, 2013
Abstract
Attrition is the Achilles' Heel of the randomized experiment: It is fairly common, and it can completely unravel the benefits of randomization. Using the structural language of causal diagrams I demonstrate that attrition is problematic for identification of the average treatment effect (ATE) if -- and only if -- it is a common effect of the treatment and the outcome (or a cause of the outcome other than the treatment). I also demonstrate that whether the ATE is identified and estimable for all units in the experiment, or only for those units with observed outcomes, depends on two d-separation conditions. One of these is testable ex-post under standard experimental assumptions. The other is testable ex-ante so long as adequate measurement protocols are adopted. Missing at Random (MAR) assumptions are neither necessary nor sufficient for identification of the ATE.
Keywords: attrition, randomized controlled experiments, field experiments, causal diagrams, directed acyclic graphs, average treatment effect, nonparametric
JEL Classification: C9, C90, C93, C99, C42
Suggested Citation: Suggested Citation