A Statistical Framework to Engage the Problem of Disengaged Survey Respondents: Measuring Public Support for Partisan Violence

Abstract

 There is growing concern across democracies that partisans are increasingly willing to support violence to resolve political disputes. Evidence for these claims come from surveys that are often plagued by disengaged respondents, who inflate estimates of fringe views. While screeners (e.g., attention checks) are commonly used to address respondent disengagement, there is no consensus on their proper use. This paper introduces a theoretical framework and statistical toolkit for surveys that use screeners to help identify disengaged respondents. Our theoretical framework is built on top of a rich literature in survey methodology that seeks to explain the cognitive processes that give rise to survey responses. Using this framework, we show that the norm of simply excluding respondents who fail a screener can introduce bias, either due to screener errors or by altering the sample composition. To address this bias, we develop a statistical toolkit for using screeners, including a newly derived partial identification estimator that adjusts for screener error. We apply this toolkit to reanalyze recent studies seeking to estimate mass support for political violence in the United States, finding that properly accounting for respondent disengagement results in lower estimated support for political violence than published estimates.