Can Online Surveys Represent the Entire Population?
31 Pages Posted: 24 Sep 2018 Last revised: 5 May 2023
There are 2 versions of this paper
Can Online Surveys Represent the Entire Population?
Abstract
A general concern with the representativeness of online surveys is that they exclude the "offline" population that does not use the internet. We run a large-scale opinion survey with (1) onliners in web mode, (2) offliners in face-to-face mode, and (3) onliners in face-to-face mode. We find marked response differences between onliners and offliners in the mixed-mode setting (1 vs. 2). Response differences between onliners and offliners in the same face-to-face mode (2 vs. 3) disappear when controlling for background characteristics, indicating mode effects rather than unobserved population differences. Differences in background characteristics of onliners in the two modes (1 vs. 3) indicate that mode effects partly reflect sampling differences. In our setting, re-weighting online-survey observations appears a pragmatic solution when aiming at representativeness for the entire population.
Keywords: offliner, mode effects, representativeness, online survey, public opinion
JEL Classification: C83, D91, I20
Suggested Citation: Suggested Citation