The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments

22 Pages Posted: 16 Feb 2017 Last revised: 26 Jul 2017

See all articles by Kirk Bansak

Kirk Bansak

University of California, San Diego

Jens Hainmueller

Stanford University - Department of Political Science; Stanford Graduate School of Business; Stanford Immigration Policy Lab

Daniel J. Hopkins

University of Pennsylvania

Teppei Yamamoto

Massachusetts Institute of Technology (MIT) - Department of Political Science

Date Written: July 25, 2017

Abstract

In recent years, political and social scientists have made increasing use of conjoint survey designs to study decision-making. Here, we study a consequential question which researchers confront when implementing conjoint designs: how many choice tasks can respondents perform before survey satisficing degrades response quality? To answer the question, we run a set of experiments where respondents are asked to complete as many as 30 conjoint tasks. Experiments conducted through Amazon's Mechanical Turk and Survey Sampling International demonstrate the surprising robustness of conjoint designs, as there are detectable but quite limited increases in survey satisficing as the number of tasks increases. Our evidence suggests that in similar study contexts researchers can assign dozens of tasks without substantial declines in response quality.

Keywords: conjoint analysis, survey experiment, survey fatigue, response bias

JEL Classification: C83

Suggested Citation

Bansak, Kirk and Hainmueller, Jens and Hopkins, Daniel J. and Yamamoto, Teppei, The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments (July 25, 2017). MIT Political Science Department Research Paper No. 2017-6, Stanford University Graduate School of Business Research Paper No. 17-15, Available at SSRN: https://ssrn.com/abstract=2916951 or http://dx.doi.org/10.2139/ssrn.2916951

Kirk Bansak

University of California, San Diego ( email )

9500 Gilman Drive
La Jolla, CA 92093
United States

Jens Hainmueller

Stanford University - Department of Political Science ( email )

Stanford, CA 94305
United States

HOME PAGE: http://www.stanford.edu/~jhain/

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Stanford Immigration Policy Lab

30 Alta Road
Stanford, CA 94305
United States

Daniel J. Hopkins

University of Pennsylvania ( email )

Stiteler Hall
Philadelphia, PA 19104
United States

HOME PAGE: http://www.danhopkins.org

Teppei Yamamoto (Contact Author)

Massachusetts Institute of Technology (MIT) - Department of Political Science ( email )

77 Massachusetts Avenue
Cambridge, MA 02139
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
510
Abstract Views
3,111
Rank
85,988
PlumX Metrics