Does Conjoint Analysis Mitigate Social Desirability Bias?

Abstract

How can we elicit truthful responses in survey research? Political scientists are often concerned about systematic measurement errors in survey questions on sensitive topics, such as gender and racial attitudes, due to social desirability bias (SDB). Conjoint analysis has become a popular tool to address this concern, despite the lack of systematic evidence that it is suitable for this purpose. In this paper, we employ a novel experimental design to investigate whether a standard fully randomized conjoint design mitigates SDB. Our experiment isolates the SDB reduction by comparing a standard conjoint design against a partially randomized design where only the socially sensitive attribute is randomly varied between the two profiles in each paired evaluation task. Our design also includes placebo conditions that are designed to remove confounding effects due to the increased attention to the varying attribute under the partial design. We implemented the proposed experiment in a survey designed to uncover respondents' attitudes toward environmental protection. We find suggestive evidence that conjoint analysis does mitigate SDB.

Keywords: response bias, survey experiment, survey methodology, conjoint analysis