Exploring Citation Count Methods of Measuring Faculty Scholarly Impact

Abstract

After US News & World Report’s announcement in 2019 that they will provide a separate ranking of law schools based on faculty scholarly impact, scrutinizing the various methods of assessing scholarly impact has been a hot topic. The various methods include reputation surveys, citation counts, and publication counts. This paper focuses on citation counts. Several methods of conducting citation counts have been circulated since the 1990s, notably Brian Leiter’s studies using Westlaw’s Law Reviews and Journals database; the Leiter study updates conducted by Gregory Sisk, et al., in 2012, 2015, and 2018; Heald and Sichelman’s look at HeinOnline and SSRN in Ranking the Academic Impact of 100 American Law Schools; and Ruhl, Vandenbergh, and Dunaway’s 2019 study using Web of Science in Total Scholarly Impact: Law Professor Citations in Non-Law Journals for interdisciplinary scholarly impact. Following the Ruhl study, faculty at Indiana University Maurer School of Law, with its strong record of interdisciplinary scholarship, were curious to learn Maurer’s overall scholarly impact.

I reviewed existing studies of law faculty scholarly impact and then conducted a study of the interdisciplinary work of the Maurer Law faculty by duplicating the Ruhl citation count method of examining law faculty publications in non-law journals. While duplicating the Ruhl method with Web of Science, and gathering citation information from HeinOnline and Google Scholar, several problems arose, from inconsistent terminology, to access issues and changes in the databases. Conducting this process indicated that a more standardized method of citation counts and measuring scholarly impact is needed. Additionally, the results illustrated that Maurer faculty are making a significant scholarly impact in interdisciplinary publications, and that a true overall scholarly impact score for a law school’s faculty must include some measure of interdisciplinary work. This article reviews existing literature on measuring scholarly impact, describes the citation count method and related issues explored at Maurer, and discusses the benefits and limitations of including interdisciplinary scholarship in evaluating law faculty scholarly impact.

Keywords: faculty scholarly impact, law schools, law school ranking, rankings, US News, citation counts, publication counts, reputation surveys, interdisciplinary scholarship, HeinOnline, SSRN, Maurer, law faculty scholarship

JEL Classification: K00