Response Order Biases in Economic Surveys

38 Pages Posted: 17 Feb 2021

See all articles by Erik Davison

Erik Davison

Baylor University

Yaqing Xiao

Capital University of Economics and Business

Hongjun Yan

DePaul University

Date Written: February 16, 2021

Abstract

Survey data are ubiquitous in the economics literature, ranging from unemployment rate and CPI to surveys of professional forecasts and consumer finances. However, their potential biases are rarely discussed. By randomizing the order of responses to the questions in an economic survey, we document a pervasive response order bias: respondents tend to select answers at or near the top of the lists, leading to a systematic bias in survey results. This bias is smaller when respondents are more certain about their answers and disappears for “objective” questions (e.g., questions on demographics and recent experiences). Our evidence directly shows the bias in the levels of many survey-based variables and indirectly implies a bias in their changes, the latter likely correlated with the uncertainty in the economy. To assess bias magnitude, we examine two salient features at the time of our survey: the COVID pandemic and the approaching 2020 presidential election. We find that respondents’ expectations are shaped by their political leanings and personal experience during the pandemic: when forecasting stock returns, GDP growth, or COVID vaccine development, respondents are more pessimistic if they lean Democratic or personally know someone with COVID, but are more optimistic if they expect their preferred presidential candidate to win. The magnitude of these effects is comparable to the size of the response order bias. The implications of our evidence for financial services, public health policy, and political elections are discussed.

Keywords: Response order effect, Survey, Anchoring, COVID, Experience effect, Partisan belief

JEL Classification: G12

Suggested Citation

Davison, Erik and Xiao, Yaqing and Yan, Hongjun, Response Order Biases in Economic Surveys (February 16, 2021). Available at SSRN: https://ssrn.com/abstract=3786894 or http://dx.doi.org/10.2139/ssrn.3786894

Erik Davison

Baylor University ( email )

School of Engineering & Computer Science
Waco, TX 76798
United States

Yaqing Xiao

Capital University of Economics and Business ( email )

Beijing
China

Hongjun Yan (Contact Author)

DePaul University ( email )

1 East Jackson Blvd.
Chicago, IL 60604
United States

HOME PAGE: http://sites.google.com/site/hongjunyanhomepage/

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
57
Abstract Views
345
rank
458,999
PlumX Metrics