Are True Scores and Construct Scores the Same? A Critical Examination of Their Substitutability and the Implications for Research Results

International Journal of Selection and Assessment, Volume 21 Number 4

16 Pages Posted: 27 Feb 2014

See all articles by Frank L. Schmidt

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business

Huy Le

University of Nevada, Las Vegas

In‐Sue Oh

Temple University - Department of Human Resource Management

Multiple version iconThere are 2 versions of this paper

Date Written: December 2013

Abstract

Relations between constructs are estimated based on correlations between measures of constructs corrected for measurement error. This process assumes that the true scores on the measure are linearly related to construct scores, an assumption that may not hold. We examined the extent to which differences in distribution shape reduce the correlation between true scores on a measure and scores on the underlying construct they are intended to measure. We found, via a series of Monte Carlo simulations, that when the actual construct distribution is normal, non-normal distributions of true scores caused this correlation to drop by an average of only .02 across 15 conditions. When both construct and true score distributions assumed different combinations of non-normal distributions, the average correlation was reduced by .05 across 375 conditions. We conclude that theory-based scales intended to measure constructs usually correlate highly with the constructs they are constructed to measure. We show that, as a result, in most cases true score correlations only modestly underestimate correlations between different constructs. However, in cases in which the two constructs are redundant, this underestimation can lead to the false conclusion that the constructs are ‘correlated but distinct constructs,’ resulting in construct proliferation.

Suggested Citation

Schmidt, Frank L. and Le, Huy and Oh, In-Sue, Are True Scores and Construct Scores the Same? A Critical Examination of Their Substitutability and the Implications for Research Results (December 2013). International Journal of Selection and Assessment, Volume 21 Number 4 . Available at SSRN: https://ssrn.com/abstract=2401159

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business ( email )

Acquisitions
5020 Main Library
Iowa City, IA 52242-1000
United States

Huy Le

University of Nevada, Las Vegas

4505 S. Maryland Parkway
Las Vegas, NV 89154
United States

In-Sue Oh (Contact Author)

Temple University - Department of Human Resource Management ( email )

1801 Liacouras Walk
Alter Hall 343
Philadelphia, PA 19122
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/in-sue-oh/

Register to save articles to
your library

Register

Paper statistics

Downloads
10
Abstract Views
254
PlumX Metrics