Comparing Personality Test Formats and Warnings: Effects on Criterion-Related Validity and Test-Taker Reactions

Abstract

The potential for applicant response distortion on personality measures remains a major concern in high-stakes testing situations. Many approaches to understanding response distortion are too transparent (e.g., instructed faking studies) or are too subtle (e.g., correlations with social desirability measures as indices of faking). Recent research reveals more promising approaches in two methods: using forced-choice (FC) personality test items and warning against faking. The present study examined effects of these two methods on criterion-related validity and test-taker reactions. Results supported incremental validity for an FC and Likert-scale measure in warning and no-warning conditions, above and beyond cognitive ability. No clear differences emerged between the FC vs Likert measures or warning vs no-warning conditions in terms of validity. However, some evidence suggested that FC measures and warnings may produce negative test-taker reactions. We conclude with implications for implementation in selection settings.