Analyzing Ordinal Data with Metric Models: What Could Possibly Go Wrong?

43 Pages Posted: 19 Nov 2015 Last revised: 17 Dec 2017

See all articles by Torrin Liddell

Torrin Liddell

Indiana University Bloomington

John Kruschke

Indiana University

Date Written: November 6, 2017

Abstract

We surveyed all articles in the Journal of Personality and Social Psychology (JPSP), Psychological Science (PS), and the Journal of Experimental Psychology: General (JEP:G) that mentioned the term "Likert," and found that 100% of the articles that analyzed ordinal data did so using a metric model. We present new evidence that analyzing ordinal data as metric is problematic. We demonstrate that treating ordinal data as metric can yield low correct detection rates, distorted effect size estimates, and greatly inflated false alarm rates. Moreover, we demonstrate that the averaging of multiple ordinal items into a Likert scale does not solve these problems. We provide examples of real data in the contexts of the difference of two groups and simple linear regression. To solve these problems we use an ordered probit model with Bayesian estimation of parameters. The ordered probit model shows appropriate correct detection rates and false alarm rates, and produces accurate effect sizes estimates and response probabilities. Bayesian estimation of this ordinal model is straight forward, yields rich and accurate information, and has no need for auxiliary sampling assumptions. We conclude that ordinal data ought to be analyzed with ordinal models, and that Bayesian estimation is an excellent method for accomplishing that goal.

Keywords: Ordinal data, Likert scales, Bayesian data analysis, false alarm rate, effect size

Suggested Citation

Liddell, Torrin and Kruschke, John, Analyzing Ordinal Data with Metric Models: What Could Possibly Go Wrong? (November 6, 2017). Available at SSRN: https://ssrn.com/abstract=2692323 or http://dx.doi.org/10.2139/ssrn.2692323

Torrin Liddell (Contact Author)

Indiana University Bloomington ( email )

Dept of Biology
100 South Indiana Ave.
Bloomington, IN 47405
United States

John Kruschke

Indiana University ( email )

Dept. of Psychological and Brain Sciences
1101 E. 10th St.
Bloomington, IN 47405
United States

HOME PAGE: http://sites.google.com/site/doingbayesiandataanalysis/about-the-author

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
1,084
Abstract Views
4,654
Rank
40,085
PlumX Metrics