Designing and Analyzing Studies that Randomize Schools to Estimate Intervention Effects on Student Academic Outcomes Without Classroom-Level Information

48 Pages Posted: 30 Apr 2011

See all articles by Pei Zhu

Pei Zhu

MDRC

Robin Jacob

University of Michigan at Ann Arbor

Howard S. Bloom

MDRC

Zeyu Xu

The Urban Institute

Date Written: April, 30 2011

Abstract

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools - which comprise three levels of clustering (students in classrooms in schools) - to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises frequently in practice because many available data sets identify the schools that students attend but not the classrooms in which they are taught. Do studies conducted under these circumstances yield results that are substantially different from what they would have been if this information had been available? The paper first considers this problem in the context of planning a school-randomized study based on preexisting two-level information about how academic outcomes for students vary across schools and across students within schools (but not across classrooms in schools). The paper next considers this issue in the context of estimating intervention effects from school-randomized studies. Findings are based on empirical analyses of four multisite data sets using academic outcomes for students within classrooms within schools. The results indicate that in almost all situations one will obtain nearly identical results whether or not the classroom or middle level is omitted when designing or analyzing studies.

Keywords: group randomized experiment, multi-level model, evaluation research and missing middle-level variance component

JEL Classification: C90, I21

Suggested Citation

Zhu, Pei and Jacob, Robin and Bloom, Howard S. and Xu, Zeyu, Designing and Analyzing Studies that Randomize Schools to Estimate Intervention Effects on Student Academic Outcomes Without Classroom-Level Information (April, 30 2011). Available at SSRN: https://ssrn.com/abstract=1826522 or http://dx.doi.org/10.2139/ssrn.1826522

Pei Zhu (Contact Author)

MDRC ( email )

200 Vesey Street
New York, NY 10281
United States

Robin Jacob

University of Michigan at Ann Arbor ( email )

500 S. State Street
Ann Arbor, MI 48109
United States

Howard S. Bloom

MDRC ( email )

200 Vesey Street
New York, NY 10281
United States

Zeyu Xu

The Urban Institute ( email )

2100 M Street, NW
Washington, DC 20037
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
39
Abstract Views
771
PlumX Metrics