Measure Twice and Cut Once: The Carpenter's Rule Still Applies

Abstract

In a lead article published by the Journal of Marketing Research in 2007, Bergkvist and Rossiter (2007) recommend that “for the many constructs in marketing that consist of a concrete singular object and a concrete attribute, such as AAd or ABrand, single-item measures should be used (page 175).” This conclusion is based on empirical analyses correlating single-item and multiple-item scales measuring attitudes towards advertisements and the advertised brands, collected at the same time in the same survey instrument. Finding no statistically significant differences between the correlations obtained with the single and multiple-items, the authors conclude that there is no loss in predictive validity with the use of single items, which is the basis for their recommendation. Obviously, their recommendation produces substantial savings in data-gathering costs. Consequently, their article has been highly cited (over 600 cites as of September 2013), by authors justifying their use of single-item measures.

In this note, I revisit well-known concepts of psychometric theory to demonstrate that this practice is ill-advised. First, I argue that repeated measures are necessary not only to improve the validity of some measurement instruments but, most importantly, to make it possible to assess and correct measurement instruments for random (non-systematic) measurement errors. Second, I argue that rather than testing for predictive validity, the authors had tested for concurrent validity, using a common survey instrument, thereby confounding their results with spurious correlations due to common-methods biases. Third, I conduct a true predictive validity test using two attitudinal scales and consumption behavior as a predictive criterion to show that, once corrected for measurement errors, multiple- item scales consistently out-perform their single-item equivalents.