Strategic Polling Limits the External Validity of Early Election Polling

Abstract

Using responses from a series of general election polls conducted between February and March of 2020, we found that a non-negligible share of participants withheld information about their true voter intention by responding strategically to hypothetical questions about potential future vote choice, thus introducing a source of total survey error. Although we propose a combination of ex-ante considerations in survey design and ex-post statistical fixes that can be used to reduce total survey error and infer true voter intention, the lack of repeated outcomes limits our ability to precisely calibrate our models. Our findings call into question the external validity of conclusions based on conditional forecasts using hypothetical polling results and are especially pertinent to early US general election forecasting, where potentially meaningless differences of a few percentage points can be misinterpreted as proof of differences in relative electability of potential party nominees during the Primary election season. We suggest that results from hypothetical election polls should instead be viewed as sources of partial information on future behavior, and interpreted based on contextual variables such as the moment in time relative to the predicted event.

Keywords: Polling, President, Strategic Polling, External Validity