Design of Web Questionnaires: The Effect of Layout in Rating Scales

Abstract:     
This article shows that respondents gain meaning from visual cues in a web survey as well as from verbal cues (words). We manipulated the layout of a five point rating scale using verbal, graphical, numerical, and symbolic language. This paper extends the existing literature in four directions: (1) all languages (verbal, graphical, numeric, and symbolic) are individually manipulated on the same rating scale, (2) a heterogeneous sample is used, (3) in which way personal characteristics and a respondent's need to think and evaluate account for variance in survey responding is analyzed, and (4) a web survey is used. Our experiments show differences due to verbal and graphical language but no effects of numeric or symbolic language are found. Respondents with a high need for cognition and a high need to evaluate are affected more by layout than respondents with a low need to think or evaluate. Furthermore, men, the elderly, and the highly educated are the most sensible for layout effects.

Number of Pages in PDF File: 35

Keywords: web survey, questionnaire lay out, context effects, need for cognition, need to evaluate

JEL Classification: C42, C81, C93

working papers series