Respondent Robotics: Simulating Responses to Likert-Scale Survey Items

28th APS Annual Convention in Chicago, IL, 2016

18 Pages Posted: 29 Dec 2018

See all articles by Jan Ketil Arnulf

Jan Ketil Arnulf

BI Norwegian Business School

Kai R. Larsen

Leeds School of Business; Information Systems Group; Gallup

Oyvind Martinsen

University of Colorado at Boulder

Date Written: May 26-29, 2016

Abstract

The semantic theory of survey responses (STSR) proposes that the prime source of statistical covariance in survey data is the degree of semantic similarity (overlap of meaning) among the items of the survey. Because semantic structures are possible to estimate using digital text algorithms, it is possible to predict the response structures of Likert-type scales a priori. The present study applies STSR in an experimental way by computing real survey responses using such semantic information. A sample of 153 randomly chosen respondents to the Multifactor Leadership Questionnaire (MLQ) was used as target. We developed an algorithm based on unfolding theory, where data from digital text analysis of the survey items served as input. Upon deleting progressive numbers (from 20%-95%) of the real responses, we let the algorithm replace these with simulated ones, and then compared the simulated datasets with the real ones. The simulated scores displayed sum score levels, alphas, and factor structures highly resembling their real origins even if up to 86% were simulated. In contrast, this was not the case when the same algorithm was operating without access to semantic information. The procedure was briefly repeated on a different measurement instrument and a different sample. This not only yielded similar results but also pointed to need for further theoretical and practical developments. Our study opens for experimental research on the effect of semantics on survey responses using computational procedures.

Keywords: semantics, simulation, surveys, semantic theory of survey response, leadership

Suggested Citation

Arnulf, Jan Ketil and Larsen, Kai R. and Martinsen, Oyvind, Respondent Robotics: Simulating Responses to Likert-Scale Survey Items (May 26-29, 2016). 28th APS Annual Convention in Chicago, IL, 2016. Available at SSRN: https://ssrn.com/abstract=3270116

Jan Ketil Arnulf (Contact Author)

BI Norwegian Business School ( email )

Nydalsveien 37
Oslo, 0442
Norway

Kai R. Larsen

Leeds School of Business; Information Systems Group ( email )

995 Regent Dr.
Boulder, CO 80309-0419
United States

Gallup ( email )

901 F St NW
Washington, DC 20004
United States

Oyvind Martinsen

University of Colorado at Boulder ( email )

1070 Edinboro Drive
Boulder, CO 80309
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
1
Abstract Views
26
PlumX Metrics