Relative Performance Measurement of Researchers: The High Impact of Data Source Selection

Abstract

This paper investigates the impact of choosing between four data sources on the relative performance measurement of scholars concerning their research output. It compares publication measures from the Social Science Citation Index, Scopus, Google Scholar, and the Handelsblatt. The study is based on data of 298 German-speaking accounting and marketing scholars. Our results show that rankings can differ considerably, and the observed correlations are in many cases even lower than those reported in previous studies are. We investigate several causes for these differences and show that data source selection generally has a higher effect than does measure selection. The results also suggest that performance measures based on Google Scholar provide information complementary to the more traditional data sources. Finally, we find differences to be more pronounced for accounting, a finding that suggests that decision makers must pay more detailed attention when using rankings in this field.

Keywords: citations, decision making, Google Scholar, Handelsblatt ranking, performance measurement, publications, ranking, research productivity, Scopus, Social Science Citation Index