Selection in Surveys

96 Pages Posted: 30 Nov 2021 Last revised: 3 Dec 2021

See all articles by Deniz Dutz

Deniz Dutz

University of Chicago

Ingrid Huitfeldt

Statistics Norway

Santiago Lacouture

affiliation not provided to SSRN

Magne Mogstad

University of Chicago

Alexander Torgovitsky

University of Chicago

Winnie van Dijk

Harvard University, Department of Economics

Multiple version iconThere are 2 versions of this paper

Date Written: December 2, 2021

Abstract

We evaluate how nonresponse affects conclusions drawn from survey data and consider how researchers can reliably test and correct for nonresponse bias. To do so, we examine a survey on labor market conditions during the COVID-19 pandemic that used randomly assigned financial incentives to encourage participation. We link the survey data to administrative data sources, allowing us to observe a ground truth for participants and nonparticipants. We find evidence of large nonresponse bias, even after correcting for observable differences between participants and nonparticipants. We apply a range of existing methods that account for nonresponse bias due to unobserved differences, including worst-case bounds, bounds that incorporate monotonicity assumptions, and approaches based on parametric and nonparametric selection models. These methods produce bounds (or point estimates) that are either too wide to be useful or far from the ground truth. We show how these shortcomings can be addressed by modeling how nonparticipation can be both active (declining to participate) and passive (not seeing the survey invitation). The model makes use of variation from the randomly assigned financial incentives, as well as the timing of reminder emails. Applying the model to our data produces bounds (or point estimates) that are narrower and closer to the ground truth than the other methods.

Suggested Citation

Dutz, Deniz and Huitfeldt, Ingrid and Lacouture, Santiago and Mogstad, Magne and Torgovitsky, Alexander and van Dijk, Winnie, Selection in Surveys (December 2, 2021). University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2021-141, Available at SSRN: https://ssrn.com/abstract=3973631 or http://dx.doi.org/10.2139/ssrn.3973631

Deniz Dutz

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

Ingrid Huitfeldt

Statistics Norway ( email )

N-0033 Oslo
Norway

Santiago Lacouture

affiliation not provided to SSRN

Magne Mogstad

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

Alexander Torgovitsky

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

Winnie Van Dijk (Contact Author)

Harvard University, Department of Economics

Cambridge, MA 02138
7736800581 (Phone)

Do you want regular updates from SSRN on Twitter?

Paper statistics

Downloads
62
Abstract Views
313
rank
404,140
PlumX Metrics