An Evaluation of Propensity Score Matching Estimators in Health Outcomes: Evidence from the Progresa/Oportunidades Program in Mexico
Posted: 5 Nov 2006
The restricted availability of experimental health data in developing countries has limited the generation of quantitative studies that involve the measurement of causal effects. Most of the information available is cross-sectional data that commonly limits the empirical analyses on health.
In this paper we assess the performance of the propensity score matching (PSM) technique as an alternative approach to approximate experimental randomization with cross-sectional data. We use data from a randomized controlled experiment for poverty alleviation in Mexico: Progresa/Oportunidades. This program represents an excellent avenue for comparisons of the PSM results with an experimental benchmark. The objective is to assess the bias of the PSM for several health outcomes: Morbidity, children under 5 * Prevalence - Any cause - Diarrhea - Respiratory infections Neonatal mortality rate Prenatal health care utilization Vaccination coverage
We generate a comparable sample of households with which to accurately evaluate the impacts of the program. We test how health measurement differences across questionnaires affect the estimated biases; and how the choice of pre-intervention variables to construct the propensity score can affect the observed differences between the experimental results and the PSM estimates. Methodology This paper uses panel data health indicators from Oportunidades participants (ENCEL 1998), and a comparison group extracted from a nationally-representative cross-sectional health survey (ENSA 1999-2000). Three different comparison samples are created to replicate, as close as possible, the characteristics of the experimental control group.
Results Health outcome measurement across surveys affects the bias between PSM and the experimental results. A rich set of pre-intervention covariates is fundamental to construct an adequate propensity score. Imposing further restrictions on sample inclusion for comparison does reduce the bias. Differences in socio-economic and epidemiological characteristics at the regional level can generate a geographic-based bias.
Keywords: Propensity score matching, randomization, health outcomes
JEL Classification: I12, C33, I30
Suggested Citation: Suggested Citation