Sample Size Requirements for Education Multi-Site RCTs that Select Sites Randomly

23 Pages Posted: 21 Apr 2017

See all articles by Robert Olsen

Robert Olsen

Rob Olsen LLC

Edward Bein

Abt Associates, Inc.

David Judkins

Abt Associates, Inc.

Date Written: April 21, 2017

Abstract

Multi-site randomized controlled trials (RCTs) in education are typically conducted in multiple schools. If the impact of the intervention varies across schools, and the schools in the RCT are not representative of the population of interest, the resulting impact estimates may be externally biased. This problem can be mitigated by selecting schools randomly.

This paper provides estimates of the number of schools required in RCTs that select schools randomly — or at least analyze the data as if schools were randomly selected — and randomize students within schools. The results show that the variation in impacts across schools, which may be very difficult to predict when planning an RCT, is a critical factor in determining the number of schools required to detect impacts of a given size.

Keywords: Randomized trial, impact heterogeneity, statistical power

JEL Classification: C93

Suggested Citation

Olsen, Robert and Bein, Edward and Judkins, David, Sample Size Requirements for Education Multi-Site RCTs that Select Sites Randomly (April 21, 2017). Available at SSRN: https://ssrn.com/abstract=2956576 or http://dx.doi.org/10.2139/ssrn.2956576

Robert Olsen (Contact Author)

Rob Olsen LLC ( email )

Media and Public Affairs Building
805 21st Street NW
Washington, DC 20052
United States
240-461-0503 (Phone)

Edward Bein

Abt Associates, Inc. ( email )

55 Wheeler Street
Cambridge, MA 02138-1168
United States

David Judkins

Abt Associates, Inc. ( email )

55 Wheeler Street
Cambridge, MA 02138-1168
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
75
Abstract Views
529
Rank
687,532
PlumX Metrics