When and Why is Attrition a Problem in Randomized Controlled Experiments and How to Diagnose it

39 Pages Posted: 20 May 2013

See all articles by Fernando Martel García

Fernando Martel García

Cambridge Social Science Decision Lab Inc.

Multiple version iconThere are 3 versions of this paper

Date Written: January 20, 2013

Abstract

Attrition is the Achilles’ Heel of the randomized experiment: it is fairly common, and it can unravel the benefits of randomization. This study considers when and why attrition is a problem, and how it can be diagnosed. The extant literature remains ambiguous because it relies on the language of probability, whereas problematic attrition depends on the underlying causal relations. This ambiguity arises because causation implies correlation but not vice versa. Using the structural causal language of directed acyclic graphs I show attrition is a problem when it is an active collider between the treatment and the outcome, or when the latent outcome is a mediator between the treatment and the attrition. Moreover, whether observed outcomes are representative of all outcomes, or only comparable across experimental arms, depends on two d-separation conditions. One of these is directly testable from the data.

Keywords: attrition, randomized controlled experiments, field experiments, causal diagrams, directed acyclic graphs, average treatment effect, nonparametric

JEL Classification: C9, C90, C93, C99, C42

Suggested Citation

Martel García, Fernando, When and Why is Attrition a Problem in Randomized Controlled Experiments and How to Diagnose it (January 20, 2013). Available at SSRN: https://ssrn.com/abstract=2267120 or http://dx.doi.org/10.2139/ssrn.2267120

Fernando Martel García (Contact Author)

Cambridge Social Science Decision Lab Inc. ( email )

Washington, DC
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
288
Abstract Views
2,383
Rank
75,192
PlumX Metrics