Beyond Baseline and Follow-Up: The Case for More T in Experiments

36 Pages Posted: 20 Apr 2016

See all articles by David J. McKenzie

David J. McKenzie

World Bank - Development Research Group (DECRG); IZA Institute of Labor Economics

Date Written: April 1, 2011

Abstract

The vast majority of randomized experiments in economics rely on a single baseline and single follow-up survey. If multiple follow-ups are conducted, the reason is typically to examine the trajectory of impact effects, so that in effect only one follow-up round is being used to estimate each treatment effect of interest. While such a design is suitable for study of highly autocorrelated and relatively precisely measured outcomes in the health and education domains, this paper makes the case that it is unlikely to be optimal for measuring noisy and relatively less autocorrelated outcomes such as business profits, household incomes and expenditures, and episodic health outcomes. Taking multiple measurements of such outcomes at relatively short intervals allows the researcher to average out noise, increasing power. When the outcomes have low autocorrelation, it can make sense to do no baseline at all. Moreover, the author shows how for such outcomes, more power can be achieved with multiple follow-ups than allocating the same total sample size over a single follow-up and baseline. The analysis highlights the large gains in power from ANCOVA rather than difference-in-differences when autocorrelations are low and a baseline is taken. The paper discusses the issues involved in multiple measurements, and makes recommendations for the design of experiments and related non-experimental impact evaluations.

Keywords: Scientific Research & Science Parks, Science Education, Statistical & Mathematical Sciences, Disease Control & Prevention, Economic Theory & Research

Suggested Citation

McKenzie, David John, Beyond Baseline and Follow-Up: The Case for More T in Experiments (April 1, 2011). World Bank Policy Research Working Paper No. 5639, Available at SSRN: https://ssrn.com/abstract=1822958

David John McKenzie (Contact Author)

World Bank - Development Research Group (DECRG) ( email )

1818 H. Street, N.W.
MSN3-311
Washington, DC 20433
United States

IZA Institute of Labor Economics ( email )

P.O. Box 7240
Bonn, D-53072
Germany

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
130
Abstract Views
746
rank
251,721
PlumX Metrics