The "Dark" Side of Engagement in Stated Choice Experiments: Exploring the Role of Time and Complexity with a Hybrid Model

Abstract

Online stated preference surveys have become increasingly common in recent years. However, concerns about respondents' engagement and data quality remain critical. Our study proposes a robust modelling framework that addresses the role of respondents' survey engagement while addressing key limitations in existing approaches. We implement a hybrid modelling framework incorporating engagement and disengagement as latent variables. The models consider engagement valence and temporal aspects while accounting for the order in which the task was displayed and individual task easiness and complexity according to the respondents' preferences. Our results show the importance of distinguishing between the time taken by respondents before and during the choice tasks, as each phase has unique implications on how respondents process the information. Moreover, the time respondents spent on the initial and later tasks has a different association with engagement and disengagement, confirming the role of ordering effects and experience learning. Finally, our results demonstrate the ambivalence property of engagement. Such results offer insight into how task design can influence respondent choices and data quality.

Keywords: survey engagement, survey disengagement, stated choice, task easiness, latent variables