Statistical Power of within and Between-Subjects Designs in Economic Experiments

CRREP working paper 2014-03

25 Pages Posted: 30 Mar 2018

See all articles by Charles Bellemare

Charles Bellemare

Université Laval - Département d'Économique; Centre Interuniversitaire sur le Risque, les Politiques Economiques et l'Emploi (CIRPÉE); IZA Institute of Labor Economics

Luc Bissonnette

Université Laval

Sabine Kröger

Université Laval - Département d'Économique

Multiple version iconThere are 3 versions of this paper

Date Written: January 2, 2014

Abstract

This paper discusses the choice of the number of participants for within-subjects (WS) designs and between-subjects (BS) designs based on simulations of statistical power allowing for different numbers of experimental periods. We illustrate the usefulness of the approach in the context of field experiments on gift exchange. Our results suggest that a BS design requires between 4 to 8 times more subjects than a WS design to reach an acceptable level of statistical power. Moreover, the predicted minimal sample sizes required to correctly detect a treatment effect with a probability of 80% greatly exceed sizes currently used in the literature. Our results suggest that adding experimental periods in an experiment can substantially increase the statistical power of a WS design, but have very little effect on the statistical power of the BS design. Finally, we discuss issues relating to numerical computation and present the powerBBK package programmed for STATA. This package allows users to conduct their own analysis of power for the different designs (WS and BS), conditional on user specified experimental parameters (true effect size, sample size, number of periods, noise levels for control and treatment, error distributions), statistical tests (parametric and nonparametric), and estimation methods (linear regression, binary choice models (probit and logit), censored regression models (tobit)).

Keywords: Within-subjects design, Between-subjects design, Sample size, Statistical power, Experiments

JEL Classification: C8, C9, D03

Suggested Citation

Bellemare, Charles and Bissonnette, Luc and Kroger, Sabine, Statistical Power of within and Between-Subjects Designs in Economic Experiments (January 2, 2014). CRREP working paper 2014-03, Available at SSRN: https://ssrn.com/abstract=3149007 or http://dx.doi.org/10.2139/ssrn.3149007

Charles Bellemare (Contact Author)

Université Laval - Département d'Économique ( email )

2325 Rue de l'Université
Ste-Foy, Quebec G1K 7P4 G1K 7P4
Canada

Centre Interuniversitaire sur le Risque, les Politiques Economiques et l'Emploi (CIRPÉE)

Pavillon De Sève
Ste-Foy, Quebec G1K 7P4
Canada

IZA Institute of Labor Economics ( email )

P.O. Box 7240
Bonn, D-53072
Germany

Luc Bissonnette

Université Laval ( email )

Université Laval, Pavillon J-A-DeSève, #2264
Québec, Quebec G1K 7P4 G1V0A6
Canada

Sabine Kroger

Université Laval - Département d'Économique ( email )

2325 Rue de l'Université
Ste-Foy, Quebec G1K 7P4 G1K 7P4
Canada

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
16
Abstract Views
270
PlumX Metrics