Reducing Participant Costs Without Sacrificing Statistical Power in Consumer Research: An Introduction to Pre-registered Interim Analysis Designs (Priads)

Abstract

The difficulty of determining how many observations to collect is a source of inefficiency in consumer behavior research. Group sequential designs, which allow researchers to perform interim analyses while data collection is ongoing, could offer a remedy. However, they are scarcely used in consumer behavior research, probably owing to low awareness, perceived complexity, or concerns about the validity of this approach. This paper offers a tutorial on group sequential designs and introduces Pre-Registered Interim Analysis Designs (PRIADs): A practical five-step procedure to facilitate the adoption of these designs in marketing. We show that group sequential designs can be easily adopted by marketing researchers, and introduce a companion app to help researchers implement them. We demonstrate multiple benefits of PRIADs for researchers engaged in confirmatory hypothesis testing: They facilitate sample size decisions, allow researchers to achieve a desired level of statistical power with a smaller number of observations, and help conduct more efficient pilot studies. We validate this cost-saving potential through a comprehensive re-analysis of 212 studies published in the Journal of Consumer Research, which shows that using PRIADs would have reduced participant costs by 20% to 29%. We conclude with a discussion of limitations and possible alternatives to PRIADs.

Keywords: experiments, experimental methods, power analysis, group sequential designs, meta-science, statistics