Social Media, Web, and Panel Surveys: Using Non-Probability Samples in Social and Policy Research

Policy & Internet, Forthcoming

27 Pages Posted: 30 Apr 2020

See all articles by Vili Lehdonvirta

Vili Lehdonvirta

University of Oxford - Oxford Internet Institute

Atte Oksanen

Tampere University - Faculty of Social Sciences

Pekka Räsänen

University of Turku

Grant Blank

University of Oxford - Oxford Internet Institute

Date Written: April 3, 2020

Abstract

The use of online surveys has grown rapidly in social science and policy research, surpassing more established methods. We argue that a better understanding is needed, especially of the strengths and weaknesses of non-probability online surveys that can be conducted relatively quickly and cheaply. We describe two common approaches to non-probability online surveys – river and panel sampling – and theorize their inherent selection biases: topical self-selection and economic self-selection. We conduct an empirical comparison of two river samples (Facebook and web-based) and one panel sample (from a major survey research company) with benchmark data grounded in a comprehensive population registry. We examine (1) how closely the online samples correspond with the benchmark, and (2) their usefulness in studying a non-demographic subpopulation. The river samples diverge from the benchmark on demographic variables and yield much higher means on non-demographic variables, even after weighting; we attribute this to topical self-selection. The panel is closer to the benchmark. When examining the characteristics of a non-demographic subpopulation, we detect no differences between the river and panel samples. We conclude that non-probability online surveys don’t replace probability surveys, but augment the researcher’s toolkit with new digital practices, such as exploratory studies of small and emerging non-demographic subpopulations.

Suggested Citation

Lehdonvirta, Vili and Oksanen, Atte and Räsänen, Pekka and Blank, Grant, Social Media, Web, and Panel Surveys: Using Non-Probability Samples in Social and Policy Research (April 3, 2020). Policy & Internet, Forthcoming , Available at SSRN: https://ssrn.com/abstract=3567823

Vili Lehdonvirta (Contact Author)

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford, Oxfordshire OX1 3JS
United Kingdom

HOME PAGE: http://www.oii.ox.ac.uk

Atte Oksanen

Tampere University - Faculty of Social Sciences ( email )

Tampere
Finland

Pekka Räsänen

University of Turku ( email )

Turku, 20014
Finland

Grant Blank

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
74
Abstract Views
611
Rank
576,524
PlumX Metrics