Differential Validity for Cognitive Ability Tests in Employment and Educational Settings: Not Much More than Range Restriction?

Journal of Applied Psychology 2014, Vol. 99, No. 1, 1-20

Fox School of Business Research Paper No. 14-007

21 Pages Posted: 27 Feb 2014 Last revised: 31 Dec 2016

See all articles by Philip L. Roth

Philip L. Roth

Clemson University - Department of Management

Huy Le

University of Nevada, Las Vegas

In‐Sue Oh

Temple University - Department of Human Resource Management

Chad H. Van Iddekinge

Florida State University - College of Business

Maury A. Buster

State of Alabama - Personnel Department

Steve Robbins

ACT Inc. - Research Division

Michael A. Campion

Purdue University - Krannert School of Management

Date Written: 2013

Abstract

The concept of differential validity suggests that cognitive ability tests are associated with varying levels of validity across ethnic groups, such that validity is lower in certain ethnic subgroups than in others. A recent meta-analysis has revived the viability of this concept. Unfortunately, data were not available in this meta-analysis to correct for range restriction within ethnic groups. We reviewed the differential validity literature and conducted 4 studies. In Study 1, we empirically demonstrated that using a cognitive ability test with a common cutoff decreases variance in test scores of Black subgroup samples more than in White samples. In Study 2, we developed a simulation that examined the effects of range restriction on estimates of differential validity. Results demonstrated that different levels of range restriction for subgroups can explain the apparent observed differential validity results in employment and educational settings (but not military settings) when no differential validity exists in the population. In Study 3, we conducted a simulation in which we examined how one corrects for range restriction affects the accuracy of these corrections. Results suggest that the correction approach using a common range restriction ratio for various subgroups may create or perpetuate the illusion of differential validity and that corrections are most accurate when done within each subgroup. Finally, in Study 4, we conducted a simulation in which we assumed differential validity in the population. We found that range restriction artificially increased the size of observed differential validity estimates when the validity of cognitive ability tests was assumed to be higher among Whites. Overall, we suggest that the concept of differential validity may be largely artifactual and current data are not definitive enough to suggest such effects exist.

Keywords: differential validity, adverse impact, personnel selection

Suggested Citation

Roth, Philip L. and Le, Huy and Oh, In-Sue and Van Iddekinge, Chad H. and Buster, Maury A. and Robbins, Steve and Campion, Michael A., Differential Validity for Cognitive Ability Tests in Employment and Educational Settings: Not Much More than Range Restriction? (2013). Journal of Applied Psychology 2014, Vol. 99, No. 1, 1-20; Fox School of Business Research Paper No. 14-007. Available at SSRN: https://ssrn.com/abstract=2401166

Philip L. Roth

Clemson University - Department of Management ( email )

101 Sirrine Hall
Clemson, SC 29634
United States
864-656-2015 (Phone)

HOME PAGE: http://business.clemson.edu/Managemt/faculty/l3_fac_Roth.html

Huy Le

University of Nevada, Las Vegas

4505 S. Maryland Parkway
Las Vegas, NV 89154
United States

In-Sue Oh (Contact Author)

Temple University - Department of Human Resource Management ( email )

1801 Liacouras Walk
Alter Hall 343
Philadelphia, PA 19122
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/in-sue-oh/

Chad H. Van Iddekinge

Florida State University - College of Business ( email )

423 Rovetta Business Building
Tallahassee, FL 32306-1110
United States

Maury A. Buster

State of Alabama - Personnel Department

64 North Union Street, Room 300
Montgomery, AL 36130
United States
334 242-3389 (Phone)
334 242-1110 (Fax)

Steve Robbins

ACT Inc. - Research Division ( email )

500 ACT Drive
Iowa City, IA 52243
United States

Michael A. Campion

Purdue University - Krannert School of Management ( email )

1310 Krannert Building
West Lafayette, IN 47907-1310
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
37
Abstract Views
474
PlumX Metrics
!

Under construction: SSRN citations will be offline until July when we will launch a brand new and improved citations service, check here for more details.

For more information