Using Eye-Tracking to Understand Decision-Making in Conjoint Experiments

Abstract

Conjoint experiments enjoy increasing popularity in political and social science, but there is a paucity of research on respondents' underlying decision-making processes. We leverage eye-tracking methodology and a conjoint experiment, administered to a subject pool consisting of university students and local community members, to examine how respondents process information when completing conjoint surveys. Our study has two main findings. First, we find a positive correlation between attribute importance measures inferred from the stated choice data and attribute importance measures based on eye movement. This validation test supports the interpretation of common conjoint metrics, such as Average Marginal Component Effects and marginal R^2 values, as valid measures of attribute importance. Second, when we experimentally increase the number of attributes and profiles in the conjoint table, respondents on average view a larger absolute number of cells but a smaller fraction of the total cells displayed, and the patterns in which they search between cells change conditionally. At the same time, however, their stated choices remain remarkably stable. This overall pattern speaks to the robustness of conjoint experiments and is consistent with a bounded rationality mechanism. Respondents can adapt to complexity by selectively incorporating relevant new information to focus on the important attributes, while ignoring less relevant information to reduce the cognitive processing costs. Together, our results have implications for both the design and interpretation of conjoint experiments.

Keywords: conjoint, survey methods, factorial design, eye tracking