A Comparison of Six Overall Evaluation Rating Scales

Abstract

A study was conducted with a random sample of 131 college students to determine whether different overall evaluation rating scales would produce different results. Six different rating scales were used to measure four stimuli (their college, a core course, the college cafeteria, and the college library). The six overall evaluation rating scales were: an "overall how would you rate" rating scale, a "likely to recommend to a friend" rating scale, a "compared to the ideal" rating scale," a "how often, if at all, do you regret having chosen/taken/used" scale, a "compared to what you expected" rating scale," and an "overall satisfaction" rating scale. The results indicate that, in general, that the "compared to ideal" and "expectations" scales generally produce more negative evaluations while the "overall rating" and "regret" scales produce more positive evaluations. Thus, a researcher might inadvertently bias a study towards the negative side by phrasing the question in terms of expectations ("Overall, compared to what you expected, how would you rate___?") or as compared to the ideal ("Compared to the ideal___, how would you rate___?"). On the other hand, phrasing the question in terms of an overall rating ("Overall, how would you rate___?") will produce more positive overall evaluations.

Keywords: rating scales, overall evaluation, ethics in research, expectation scales, measurement bias, likely to recommend scale

JEL Classification: C00, C10, C40, I20