Assessing Three Sources of Misresponse to Reversed Likert Items

Abstract

Data collected using multi-item Likert scales that contain reversed items often exhibit problems, such as unexpected factor structures and diminished scale reliabilities. These problems arise when respondents select responses on the same side of the scale neutral point for both reversed and non-reversed items, a phenomenon we call misresponse. Across four experiments and an exploratory study using published data, the authors find that misresponse to reversed Likert items averaged nearly 20%, twice the level identified as problematic in previous simulation studies. Counter to prevailing thought, the patterns of misresponse and response latency across manipulated items could not be attributed to respondent inattention or acquiescence. Instead, the pattern supports an item verification difficulty explanation, which holds that task complexity, and thus misresponse and response latency, increases with the number of cognitive operations required for a respondent to compare a scale item with his or her belief. The observed results are well explained by the Constituent Comparison Model.

Keywords: measurement, Likert, acquiescence, inattention, response styles, reversed items, reverse-scaled, negation, psycholinguistics, constituent comparison

JEL Classification: M31, C42