Practical Methods for Assessing the Quality of Subjective Selection Processes

Abstract

Selection processes are everywhere in business and society: new product development, college admissions, hiring, and even academic journal submissions. Information on candidates is typically combined in a subjective or holistic manner, making assessment of the quality of the process challenging. In this paper, we address the question, “how can we determine the effectiveness of a selection process?” We show that even if selection is subjective, we can evaluate the process by measuring an additional audit variable that is at least somewhat predictive of performance. This approach can be used either with or without observing eventual performance. We illustrate our methods with data from two commercial settings in which new product opportunities are selected.

Keywords: selection, evaluation, filter, assessment, innovation, hiring, recruiting, admissions

JEL Classification: C4, D8