Retrospective vs. Prospective Analyses of School Inputs: The Case of Flip Charts in Kenya

Abstract

This paper compares retrospective and prospective analyses of the effect of flip charts on test scores in rural Kenyan schools. Retrospective estimates that focus on subjects for which flip charts are used suggest that flip charts raise test scores by up to 20 percent of a standard deviation. Controlling for other educational inputs does not reduce this estimate. In contrast, prospective estimators based on a study of 178 schools, half of which were randomly selected to receive charts, provide no evidence that flip charts increase test scores. One interpretation is that the retrospective results were subject to omitted variable bias despite the inclusion of control variables. If the direction of omitted variable bias were similar in other retrospective analyses of educational inputs in developing countries, the effects of inputs may be even more modest than retrospective studies suggest. Bias appears to be reduced by a differences-in-differences estimator that examines the impact of flip charts on the relative performance of students in flip chart and other subjects across schools with and without flip charts, but it is not clear that this approach is applicable more generally.