Sample Size Requirements for Education Multi-Site RCTs that Select Sites Randomly

Abstract

Multi-site randomized controlled trials (RCTs) in education are typically conducted in multiple schools. If the impact of the intervention varies across schools, and the schools in the RCT are not representative of the population of interest, the resulting impact estimates may be externally biased. This problem can be mitigated by selecting schools randomly.

This paper provides estimates of the number of schools required in RCTs that select schools randomly — or at least analyze the data as if schools were randomly selected — and randomize students within schools. The results show that the variation in impacts across schools, which may be very difficult to predict when planning an RCT, is a critical factor in determining the number of schools required to detect impacts of a given size.

Keywords: Randomized trial, impact heterogeneity, statistical power

JEL Classification: C93