Analyzing Ordinal Data with Metric Models: What Could Possibly Go Wrong?

43 Pages Posted: 19 Nov 2015 Last revised: 17 Dec 2017

Torrin Liddell

Indiana University Bloomington

John Kruschke

Indiana University

Date Written: November 6, 2017

Abstract

We surveyed all articles in the Journal of Personality and Social Psychology (JPSP), Psychological Science (PS), and the Journal of Experimental Psychology: General (JEP:G) that mentioned the term "Likert," and found that 100% of the articles that analyzed ordinal data did so using a metric model. We present new evidence that analyzing ordinal data as metric is problematic. We demonstrate that treating ordinal data as metric can yield low correct detection rates, distorted effect size estimates, and greatly inflated false alarm rates. Moreover, we demonstrate that the averaging of multiple ordinal items into a Likert scale does not solve these problems. We provide examples of real data in the contexts of the difference of two groups and simple linear regression. To solve these problems we use an ordered probit model with Bayesian estimation of parameters. The ordered probit model shows appropriate correct detection rates and false alarm rates, and produces accurate effect sizes estimates and response probabilities. Bayesian estimation of this ordinal model is straight forward, yields rich and accurate information, and has no need for auxiliary sampling assumptions. We conclude that ordinal data ought to be analyzed with ordinal models, and that Bayesian estimation is an excellent method for accomplishing that goal.

Keywords: Ordinal data, Likert scales, Bayesian data analysis, false alarm rate, effect size

Suggested Citation

Liddell, Torrin and Kruschke, John, Analyzing Ordinal Data with Metric Models: What Could Possibly Go Wrong? (November 6, 2017). Available at SSRN: https://ssrn.com/abstract=2692323 or http://dx.doi.org/10.2139/ssrn.2692323

Torrin Liddell (Contact Author)

Indiana University Bloomington ( email )

100 South Indiana Ave.
Bloomington, IN 47405
United States

John Kruschke

Indiana University ( email )

Dept. of Psychological and Brain Sciences
1101 E. 10th St.
Bloomington, IN 47405
United States

HOME PAGE: http://sites.google.com/site/doingbayesiandataanalysis/about-the-author

Register to save articles to
your library

Register

Paper statistics

Downloads
267
rank
105,922
Abstract Views
1,214
PlumX