Knowing What Works: The Case for Rigorous Program Evaluation

Abstract

Since interventions by the public sector generally commit substantial societal resources, the evaluation of effects and costs of policy interventions is imperative. This paper outlines why program evaluation should follow well-respected scientific standards and why it should be performed by independent researchers. Moreover, it outlines the three fundamental elements of evaluation research, the choice of the appropriate outcome measure, the assessment of the direct and indirect cost associated with the intervention, and the attribution of effects to underlying causes. The paper proceeds to outline in intuitive terms that the construction of a credible counterfactual situation is at the heart of the formal statistical evaluation problem. It introduces several approaches, based on both experiments and on non-experimental data, that have been proposed in the literature to solve the evaluation problem, and illustrates them numerically.

Keywords: counterfactual, observational studies, Experiments

JEL Classification: H43, C40, C90