Getting the Race Wrong: Sampling Biases and Black Voters in Online, Opt-In Polls

25 Pages Posted: 31 Oct 2023

See all articles by Daniel J. Hopkins

Daniel J. Hopkins

University of Pennsylvania

William Halm

University of Pennsylvania - Department of Political Science

Melissa Huerta

University of Pennsylvania

Josearmando Torres

University of Pennsylvania

Date Written: October 3, 2023

Abstract

Social scientists are increasingly reliant on online, opt-in surveys. Such samples enable researchers to conduct affordable experiments with results that often generalize. But can these surveys effectively represent Black respondents? This paper presents the results of surveys conducted online and in person using multiple samples in a racially diverse American city--Philadelphia--during its 2023 mayoral primary. The participation rate for online surveys promoted via Facebook and Instagram is just 0.4%, with White residents and those with college degrees markedly more likely to respond. Such biases help explain why neither our surveys nor public polls correctly identified the primary's ultimate winner, an establishment-backed Black Democrat. Even weighted, geographically stratified online surveys typically under-estimate the winner's support, although an in-person exit poll over-estimates her support. These results indicate important gaps in the populations represented in contemporary opt-in surveys and suggest that supplemental surveys via alternative modes may help reduce those gaps.

Keywords: public opinion, Black Americans, survey research, online surveys, opt-in surveys

JEL Classification: H00, H80

Suggested Citation

Hopkins, Daniel J. and Halm, William and Huerta, Melissa and Torres, Josearmando, Getting the Race Wrong: Sampling Biases and Black Voters in Online, Opt-In Polls (October 3, 2023). Available at SSRN: https://ssrn.com/abstract=4591422 or http://dx.doi.org/10.2139/ssrn.4591422

Daniel J. Hopkins (Contact Author)

University of Pennsylvania ( email )

Stiteler Hall
Philadelphia, PA 19104
United States

HOME PAGE: http://www.danhopkins.org

William Halm

University of Pennsylvania - Department of Political Science ( email )

Stiteler Hall
Philadelphia, PA 19104
United States

Melissa Huerta

University of Pennsylvania ( email )

Philiadelphia, PA
United States

Josearmando Torres

University of Pennsylvania ( email )

Philiadelphia, PA
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
54
Abstract Views
280
Rank
636,975
PlumX Metrics