National Bureau of Economic Research (NBER); University of Michigan at Ann Arbor - Gerald R. Ford School of Public Policy; University of Michigan at Ann Arbor - School of Education
Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. Our collaborative network of five research teams applying this design in early education and methods experts has identified six challenges that need to be carefully considered in this next context: 1) available baseline covariates may not be very rich; 2) limited data on the counterfactual; 3) limited and inconsistent outcome data; 4) weakened internal validity due to attrition; 5) constrained external validity due to who competes for oversubscribed programs; and 6) difficulties answering site-level questions with child-level randomization. We offer potential solutions to these six challenges and concrete recommendations for the design of future lottery-based early education studies.
Weiland, Christina and Unterman, Rebecca and Dynarski, Susan M. and Dynarski, Susan M. and Abenavoli, Rachel and Bloom, Howard S. and Braga, Breno and Faria, Ann-Marie and Greenberg, Erica H. and Jacob, Brian A. and Arnold Lincove, Jane and Manship, Karen and McCormick, Meghan and Miratrix, Luke and Monarrez, Tomas and Morris-Perez, Pamela and Shapiro, Anna and Valant, Jon and Bell Weixler, Lindsay, Lottery-Based Evaluations of Early Education Programs: Opportunities and Challenges for Building the Next Generation of Evidence (February 2023). NBER Working Paper No. w30970, Available at SSRN: https://ssrn.com/abstract=4364718