Getting the Race Wrong: A Case Study of Sampling Bias and Black Voters in Online, Opt-In Polls

30 Pages Posted: 31 Oct 2023 Last revised: 23 May 2024

See all articles by Daniel J. Hopkins

Daniel J. Hopkins

University of Pennsylvania

William Halm

University of Pennsylvania - Department of Political Science

Melissa Huerta

University of Pennsylvania

Josearmando Torres

University of Pennsylvania

Date Written: May 23, 2024

Abstract

Researchers are increasingly reliant on online, opt-in surveys. But prior benchmarking exercises employ national samples, making it unclear whether such surveys can effectively represent Black respondents and other minorities nationwide. This paper presents the results of uncompensated online and in-person surveys administered chiefly in one racially diverse American city--Philadelphia--during its 2023 mayoral primary. The participation rate for online surveys promoted via Facebook and Instagram was 0.4%, with White residents and those with college degrees more likely to respond. Such biases help explain why neither our surveys nor public polls correctly identified the Democratic primary's winner, an establishment-backed Black Democrat. Even weighted, geographically stratified online surveys typically under-estimate the winner's support, although an in-person exit poll does not. We identify some similar patterns in Chicago. These results indicate important gaps in the populations represented in contemporary opt-in surveys and suggest that alternative survey modes help reduce them.

Keywords: public opinion, Black Americans, survey research, online surveys, opt-in surveys

JEL Classification: H00, H80

Suggested Citation

Hopkins, Daniel J. and Halm, William and Huerta, Melissa and Torres, Josearmando, Getting the Race Wrong: A Case Study of Sampling Bias and Black Voters in Online, Opt-In Polls (May 23, 2024). Available at SSRN: https://ssrn.com/abstract=4591422 or http://dx.doi.org/10.2139/ssrn.4591422

Daniel J. Hopkins (Contact Author)

University of Pennsylvania ( email )

Stiteler Hall
Philadelphia, PA 19104
United States

HOME PAGE: http://www.danhopkins.org

William Halm

University of Pennsylvania - Department of Political Science ( email )

Stiteler Hall
Philadelphia, PA 19104
United States

Melissa Huerta

University of Pennsylvania ( email )

Philiadelphia, PA
United States

Josearmando Torres

University of Pennsylvania ( email )

Philiadelphia, PA
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
191
Abstract Views
1,423
Rank
342,312
PlumX Metrics