Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies

37 Pages Posted: 20 Apr 2016

See all articles by Talip Kilic

Talip Kilic

World Bank - Development Data Group (DECDG)

Thomas Pave Sohnesen

World Bank

Multiple version iconThere are 2 versions of this paper

Date Written: January 1, 2015

Abstract

Does the same question asked of the same population yield the same answer in face-to-face interviews when other parts of the questionnaire are altered? If not, what would be the implications for proxy-based poverty measurement? Relying on a randomized household survey experiment implemented in Malawi, this study finds that observationally equivalent as well as same households answer the same questions differently when interviewed with a short questionnaire versus the longer counterpart that, in a prior survey round, would have informed the prediction model for a proxy-based poverty measurement exercise. The analysis yields statistically significant differences in reporting between the short and long questionnaires across all topics and types of questions. The reporting differences result in significantly different predicted poverty rates and Gini coefficients. While the difference in predictions ranges from approximately 3 to 7 percentage points depending on the model specification, restricting the proxies to those collected prior the variation in questionnaire design, namely demographic variables from the household roster and location fixed effects, leads to same predictions in both samples. The findings emphasize the need for further methodological research, and suggest that short questionnaires designed for proxy-based poverty measurement should be piloted, prior to implementation, in parallel with the longer questionnaire from which they have evolved. The fact that at the median it took 25 minutes to complete the food and non-food consumption sections in the long questionnaire also implies that the implementation of these sections might not be as overly costly as usually assumed.

Keywords: Poverty Diagnostics, Policy Formulation and Assessment (superceded), Democratic Government, Small Area Estimation Poverty Mapping, Public Sector Administrative and Civil Service Reform, Poverty Monitoring & Analysis, Poverty Assessment, Organizational Management, Public Sector Administrative & Civil Service Reform, Public Sector Regulation, De Facto Governments, Poverty Impact Evaluation, Poverty Lines

Suggested Citation

Kilic, Talip and Sohnesen, Thomas Pave, Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies (January 1, 2015). World Bank Policy Research Working Paper No. 7182. Available at SSRN: https://ssrn.com/abstract=2579881

Talip Kilic (Contact Author)

World Bank - Development Data Group (DECDG) ( email )

Via Labicana 110
Rome, Lazio 00184
Italy

Thomas Pave Sohnesen

World Bank ( email )

1818 H Street, N.W.
Washington, DC 20433
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
36
Abstract Views
445
PlumX Metrics