'Pyramiding': Efficient Identification of Rare Subjects

Abstract

The need to economically identify rare subjects within large, poorly-mapped search spaces is a frequently-encountered problem for social scientists and managers. It is notoriously difficult, for example, to identify the best new CEO for our company, or the best three lead users to participate in our product development project. Mass screening of entire populations or samples becomes steadily more expensive as the number of acceptable solutions within the search space becomes rarer. Pyramiding can be significantly more efficient under many conditions.

Pyramiding is a search process based upon the view that people with a strong interest in a topic or field tend to know people more expert than themselves. In this paper we empirically explore the efficiency of pyramiding searches relative to mass screening in search spaces where there is only one best solution. In four experiments, we find that pyramiding in each case identifies the best solution within the search space using an average of only 30% of the effort required by mass screening. Based on our findings, we propose conditions under which pyramiding will be a more efficient approach to exploring a search space than screening.

Keywords: pyramiding