The Mixing Problem in Program Evaluation

Abstract

A common concern of evaluation studies is to learn the distribution of outcomes when a specified treatment policy or assignment rule, determines the treatment received by each member of a specified population. Recent studies have emphasized evaluation of policies providing the same treatment to all members of the population. In particular, experiments with randomized treatments have this objective. Policies mandating homogenous treatment of the population are of interest, but so are ones that permit treatment to vary across the population. This paper examines the use of empirical evidence on programs with homogenous treatments to infer the outcomes that would occur if treatment were to vary across the population. Experimental evidence from the Perry Preschool Project is used to illustrate the inferential problem and the main findings of the analysis.