From Local to Global: External Validity in a Fertility Natural Experiment
84 Pages Posted: 23 Sep 2015
There are 4 versions of this paper
From Local to Global: External Validity in a Fertility Natural Experiment
From Local to Global: External Validity in a Fertility Natural Experiment
From Local to Global: External Validity in a Fertility Natural Experiment
From Local to Global: External Validity in a Fertility Natural Experiment
Date Written: August 22, 2015
Abstract
Experimental evidence on a range of interventions in developing countries is accumulating rapidly. Is it possible to extrapolate from an experimental evidence base to other locations of policy interest (from “reference” to “target” sites)? And which factors determine the accuracy of such an extrapolation? We investigate applying the Angrist and Evans (1998) natural experiment (the effect of boy-boy or girl-girl as the first two children on incremental fertility and mothers’ labor force participation) to data from International IPUMS on 166 country-year censuses. We define the external validity function with extrapolation error depending on covariate differences between reference and target locations, and find that smaller differences in geography, education, calendar year, and mothers’ labor force participation lead to lower extrapolation error. As experimental evidence accumulates, out-of-sample extrapolation error does not systematically approach zero if the available evidence base is naïvely extrapolated, but does if the external validity function is used to select the most appropriate reference context for a given target (although absolute error remains meaningful relative to the magnitude of the treatment effect). We also investigate where to locate experiments and the decision problem associated with extrapolating from existing evidence rather than running a new experiment at a target site.
Keywords: external validity, fertility, labor supply, experiments
JEL Classification: C930, D040, D100, O120
Suggested Citation: Suggested Citation