Automated Chat Application Surveys Using Whatsapp: Evidence from Panel Surveys and a Mode Experiment

51 Pages Posted: 19 May 2022

See all articles by Jennifer Fei

Jennifer Fei

Stanford University

Jessica Wolff

Stanford University

Michael Hotard

Stanford University

Hannah Ingham

Slalom Consulting

Saurabh Khanna

Stanford University

Duncan Lawrence

Immigration Policy Lab, Stanford University

Beza Tesfaye

Mercy Corps

Jeremy M. Weinstein

Stanford University - Freeman Spogli Institute for International Studies; Center for Global Development

Vasil I. Yasenov

Immigration Policy Lab

Jens Hainmueller

Stanford University - Department of Political Science; Stanford Graduate School of Business; Stanford Immigration Policy Lab

Abstract

We present a method to conduct automated surveys over WhatsApp, a globally popular messaging service. WhatsApp surveys offer potential advantages since they incur relatively low costs to respondents and researchers, are easy to use for respondents who are already familiar with WhatsApp chat features, and facilitate continued engagement with mobile populations as users can retain their WhatsApp number even if they change SIM cards and phone numbers. Yet, there is limited knowledge on how well WhatsApp surveys perform empirically. We test the WhatsApp method using two original panel surveys of refugees in Colombia and the U.S. and find satisfactory response rates and retention over a nine-month follow-up period in these mobile populations. We also conduct a mode experiment in Colombia comparing WhatsApp to short message service (SMS) and interactive voice response (IVR) surveys. We find that WhatsApp had a 12 and 27 percentage points higher response rate than IVR and SMS, respectively, resulting from higher initial engagement and lower break-off. We conclude by discussing advantages and limitations of the WhatsApp method and offer documentation and a public code repository to support researchers and practitioners in applying the method in other contexts.

Keywords: survey methods, panel data, mobile populations, mode experiment

JEL Classification: F22, J15, K42

Suggested Citation

Fei, Jennifer and Wolff, Jessica and Hotard, Michael and Ingham, Hannah and Khanna, Saurabh and Lawrence, Duncan and Tesfaye, Beza and Weinstein, Jeremy M. and Yasenov, Vasil and Hainmueller, Jens, Automated Chat Application Surveys Using Whatsapp: Evidence from Panel Surveys and a Mode Experiment. IZA Discussion Paper No. 15263, Available at SSRN: https://ssrn.com/abstract=4114839 or http://dx.doi.org/10.2139/ssrn.4114839

Jennifer Fei (Contact Author)

Stanford University

Stanford, CA 94305
United States

Jessica Wolff

Stanford University

Stanford, CA 94305
United States

Michael Hotard

Stanford University

Hannah Ingham

Slalom Consulting

7800 Forsyth Boulevard Suite 850
St. Louis, MO 63105
United States

Saurabh Khanna

Stanford University

Stanford, CA 94305
United States

Duncan Lawrence

Immigration Policy Lab, Stanford University ( email )

616 Serra St.
Stanford, CA 94305
United States

HOME PAGE: http://www.immigrationlab.org

Beza Tesfaye

Mercy Corps

1111 19th Street NW
Suit 650
Washington, DC 20036

Jeremy M. Weinstein

Stanford University - Freeman Spogli Institute for International Studies ( email )

Stanford, CA 94305
United States

Center for Global Development

2055 L St. NW
5th floor
Washington, DC 20036
United States

Vasil Yasenov

Immigration Policy Lab ( email )

Stanford, CA 94305
United States

Jens Hainmueller

Stanford University - Department of Political Science ( email )

Stanford, CA 94305
United States

HOME PAGE: http://www.stanford.edu/~jhain/

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Stanford Immigration Policy Lab

30 Alta Road
Stanford, CA 94305
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
148
Abstract Views
718
Rank
371,883
PlumX Metrics