Sample Calibration in Likert-Metric Survey Data

Abstract

Likert scales are a widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale. The current research develops a Fisher Information metric for Likert scales, and explores the effect of particular survey design decisions or results on the information content and loss of Likert data. A Fisher Information metric outperforms earlier metrics by converging reliably to values that are intuitive in the sense that they suggest that information captured from subjects is fairly stable. The results of the analysis suggest that, varying bias and response dispersion inherent in specific surveys may require increases of sample size by several orders of magnitude to compensate for information loss and in order to derive valid conclusions at a given significance and power of tests. A prioritization of quality of design, and the factors relevant to quality of Likert scaled survey design is presented in the conclusions, and illustrative examples provide insight and guidance to the assessment of information content in a survey.

Keywords: psychometrics, Likert scale, information theory, survey, economics, mathematical models

JEL Classification: C39