How Much Does Teacher Quality Vary Across Teacher Preparation Programs? Reanalyzing Estimates from 6 States
48 Pages Posted: 28 Jun 2017 Last revised: 25 Oct 2017
Date Written: June 20, 2017
At least sixteen US states have begun to hold teacher preparation programs (TPPs) accountable for teacher value-added to student test scores. Yet it is unclear whether the teacher quality differences between TPPs are large enough to make an accountability system worthwhile. We reanalyze the results of peer-reviewed TPPs evaluations in 6 states: New York, Louisiana, Missouri, Washington, Texas, and Florida. Our methods are implemented in our new caterpillar command for Stata. Our results show that teacher quality differences between most TPPs are negligible — .01-.03 standard deviations in student test scores — even in states where larger differences were reported originally. Several statistical practices can exaggerate apparent differences between TPPs, including outliers, underestimation of standard errors (SEs), display of narrow confidence intervals that extend only one SE in each direction, failure to correct for multiple comparisons, underappreciation of how noise affects the distribution of TPP estimates, and confounding of between-TPP variance with variance in the comparison group. Once these issues are addressed, in some states it is possible to single out 1 or 2 TPPs whose teachers are significantly above or below average, though typically only in one subject.
Keywords: accountability, heterogeneity, teacher training
JEL Classification: I21, I28, C1
Suggested Citation: Suggested Citation