The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments
MIT Political Science Department Research Paper No. 2017-6
Stanford University Graduate School of Business Research Paper No. 17-15
22 Pages Posted: 16 Feb 2017 Last revised: 26 Jul 2017
Date Written: July 25, 2017
Abstract
In recent years, political and social scientists have made increasing use of conjoint survey designs to study decision-making. Here, we study a consequential question which researchers confront when implementing conjoint designs: how many choice tasks can respondents perform before survey satisficing degrades response quality? To answer the question, we run a set of experiments where respondents are asked to complete as many as 30 conjoint tasks. Experiments conducted through Amazon's Mechanical Turk and Survey Sampling International demonstrate the surprising robustness of conjoint designs, as there are detectable but quite limited increases in survey satisficing as the number of tasks increases. Our evidence suggests that in similar study contexts researchers can assign dozens of tasks without substantial declines in response quality.
Keywords: conjoint analysis, survey experiment, survey fatigue, response bias
JEL Classification: C83
Suggested Citation: Suggested Citation