Evaluating Journal Quality and the Association for Information Systems Senior Scholars’ Journal Basket via Bibliometric Measures: Do Expert Journal Assessments Add Value?

MIS Quarterly Vol. 37(4), 993–1012

63 Pages Posted: 9 Dec 2012 Last revised: 8 Jan 2018

See all articles by Paul Benjamin Lowry

Paul Benjamin Lowry

Virginia Polytechnic Institute & State University - Pamplin College of Business

Gregory D Moody

University of Nevada, Las Vegas - College of Business

James Gaskin

Brigham Young University - Marriott School; Case Western Reserve University - Department of Information Systems

Dennis Galletta

University of Pittsburgh and Director, Katz Doctoral Program

Sean Humphreys

Brigham Young University - Department of Information Systems

Jordan B. Barlow

California State University, Fullerton - Mihaylo College of Business & Economics

David Wilson

University of Arizona - Department of Management Information Systems

Date Written: 2013

Abstract

Information systems (IS) journal rankings and ratings help scholars focus their publishing efforts and are widely used surrogates for judging the quality of research. Over the years, numerous approaches have been used to rank IS journals, approaches such as citation metrics, school lists, acceptance rates, and expert assessments. However, the results of these approaches often conflict due to a host of validity concerns. In the current scientometric study, we make significant strides toward correcting for these limitations in the ranking of mainstream IS journals. We compare expert rankings to bibliometric measures such as the ISI Impact Factor™, the h-index, and social network analysis metrics. Among other findings, we conclude that bibliometric measures provide very similar results to expert-based methods in determining a tiered structure of IS journals, thereby suggesting that bibliometrics can be a complete, a less expensive, and a more efficient substitute for expert assessment. We also find strong support for seven of the eight journals in the Association for Information Systems (AIS) Senior Scholars’ “basket” of journals. A cluster analysis of our results indicates a two-tiered separation in the quality of the highest quality IS journals — with MISQ, ISR, and JMIS belonging, in that order, to the highest A tier. Journal quality metrics fit nicely into the sociology of science literature and can be useful in models that attempt to explain how knowledge disseminates through scientific communities.

Keywords: Information systems journal rankings, scientometrics, bibliometrics, journal quality, SenS-6, SenS-8, self-citation, Impact Factor, h-index, social network analysis, expert opinion, composite ranking or rating, AIS Senior Scholars basket of journals, nomologies for dissemination of scientific knowle

Suggested Citation

Lowry, Paul Benjamin and Moody, Gregory Daniel and Gaskin, James and Galletta, Dennis and Humphreys, Sean and Barlow, Jordan B. and Wilson, David, Evaluating Journal Quality and the Association for Information Systems Senior Scholars’ Journal Basket via Bibliometric Measures: Do Expert Journal Assessments Add Value? (2013). MIS Quarterly Vol. 37(4), 993–1012. Available at SSRN: https://ssrn.com/abstract=2186798

Paul Benjamin Lowry (Contact Author)

Virginia Polytechnic Institute & State University - Pamplin College of Business ( email )

1016 Pamplin Hall
Blacksburg, VA 24061
United States

Gregory Daniel Moody

University of Nevada, Las Vegas - College of Business ( email )

United States

HOME PAGE: http://https://faculty.unlv.edu/wpmu/gmoody/

James Gaskin

Brigham Young University - Marriott School ( email )

United States

Case Western Reserve University - Department of Information Systems ( email )

United States

Dennis Galletta

University of Pittsburgh and Director, Katz Doctoral Program ( email )

135 N Bellefield Ave
Pittsburgh, PA 15260
United States

Sean Humphreys

Brigham Young University - Department of Information Systems ( email )

510 Tanner Building
Marriott School
Provo, UT 84602
United States

Jordan B. Barlow

California State University, Fullerton - Mihaylo College of Business & Economics ( email )

P.O. Box 6848
Fullerton, CA CA 92834-6848
United States

David Wilson

University of Arizona - Department of Management Information Systems ( email )

AZ
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
650
rank
36,996
Abstract Views
3,116
PlumX Metrics