Mode Effects in Mixed-Mode Economic Surveys: Insights from a Randomized Experiment

FEDS Working Paper No. 2015-008

http://dx.doi.org/10.17016/FEDS.2015.008

36 Pages Posted: 6 Mar 2015

See all articles by Joanne W. Hsu

Joanne W. Hsu

Board of Governors of the Federal Reserve System

Brooke Helppie-McFall

University of Michigan, Survey Research Center

Date Written: February 5, 2015

Abstract

Web-based surveys have become increasingly common in economic, marketing, and other social science research. However, questions exist about the comparability of data gathered using a web interview and data gathered using more traditional survey modes, particularly for surveys on household economic behavior. Differences between data from different survey modes may arise through two different mechanisms: sample selectivity due to (lack of) web access and mode effects. This study leverages the randomized experimental design of the mixed-mode Cognitive Economics Study to examine mode effects separately from sample selectivity issues. In particular, we examine differences in survey response rates, item nonresponse, and data quality due to mode effects. Our results indicate that, in contrast to mail mode, web mode surveys (1) attain higher response rates among web users, (2) display lower item nonresponse, and (3) elicit more precise values for financial measures. We conclude that, for web-using populations, web mode surveys appear to result in more usable data than mail mode surveys, and these data appear to be of high quality. However, we also find no systematic mode differences in the categorical distributions of responses to items, providing no evidence that pooling data from the two modes is inadvisable.

Keywords: data quality, household surveys, mode effects, response rates

JEL Classification: C83, C81, C42

Suggested Citation

Hsu, Joanne W. and McFall, Brooke Helppie, Mode Effects in Mixed-Mode Economic Surveys: Insights from a Randomized Experiment (February 5, 2015). FEDS Working Paper No. 2015-008. Available at SSRN: https://ssrn.com/abstract=2571447 or http://dx.doi.org/10.2139/ssrn.2571447

Joanne W. Hsu (Contact Author)

Board of Governors of the Federal Reserve System ( email )

20th Street and Constitution Avenue NW
Washington, DC 20551
United States

Brooke Helppie McFall

University of Michigan, Survey Research Center ( email )

426 Thompson St
Ann Arbor, MI 48104
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
30
Abstract Views
273
PlumX Metrics