Once a Satisficer, Not Always a Satisficer: Evidence from an Awareness Control Experiment
18 Pages Posted: 22 Aug 2014
Date Written: 2014
Satisficing is a well-known data quality problem in all types of survey research. In this digital age however, the researcher, when using web surveys, is able to monitor and intervene in the respondent’s response process to a greater extent. In this research note, we aim to detect satisficing behavior by respondents early in a survey to see whether this response behavior continues throughout the survey and whether this behavior can be remedied. We detect satisficers by introducing an “awareness control” question that has the same outline as other attitude questions, but instead of asking for a response on an attitude question, it requests the respondent to return a specific answer (e.g. “2”) on the specified scale. We also introduce two questions asking the respondent to motivate two responses they have given previously in the survey with the aim to increase the respondents’ attentiveness to the questions and perhaps decrease future satisficing behavior.
Using a 3x2 full factorial survey embedded experiment on 1,954 respondents, our study compares data quality between six groups: one control group not receiving the awareness control, one receiving it, and, one receiving it and also being notified if they failed and asked to retake it. These three groups later either get two requests to motivate their previous responses or not. The results show that introducing an awareness control does not increase data quality, and neither does asking the respondent to motivate their answers. Most importantly though, in contrast to previous beliefs, satisficing behavior amongst respondents at one part of the survey, does not predict satisficing in other parts of the survey.
Keywords: Satisficing, Awareness Control, Instructional Manipulation Check, Web Survey
Suggested Citation: Suggested Citation