Analyzing the Impact of Events Through Surveys: Formalizing Biases and Introducing the Dual Randomized Survey Design

Abstract

Social scientists and public opinion analysts often use survey data to test how important events impact respondent beliefs, attitudes, and preferences. This paper offers a formal analysis of the pre-event/post-event survey approach, including designs that seek to reduce bias using quota sampling, rolling cross-sections, and panels. Our analysis distinguishes between various sources of bias and clarifies the comparative strengths and weaknesses of each approach. We then propose a modified panel design that can reduce bias in cases where asking respondents to complete the same survey twice could impact their responses in Wave 2. This issue is acute when fielding conventional pre-event/post-event panels due to the short time horizon between Waves 1 and 2. Our analysis elucidates important insights that can improve social scientists’ ability to study the causal impact of important events through surveys.

Keywords: Surveys, Causal Inference, Potential Outcomes Framework, Quota Sampling, Rolling Cross-Sections, Panels