Response Incomparability in Self-Reported Mobility

Abstract

Rationale: Measuring health in populations is an essential component of health system evaluation and intervention assessment. A central element in most health measurement strategies remains self-reported responses to categorical items in population surveys. Recent work has indicated that, even when identical or equivalent items are used in surveys, results may not be comparable across populations. This phenomenon of response incomparability has been demonstrated thus far mostly by evaluation of face validity or comparison to other population-level indicators. Evidence of response incomparability at the individual level remains limited.

Objectives: The objective of this study was to examine how self-reported mobility varies across individuals having the same measured level of mobility.

Methodology: We used data from a multi-country survey study in which a subset of respondents were administered the posturo-locomotion-manual (PLM) test of mobility as well as responding to 5 self-reported mobility items. Variation in self reports was assessed across countries, by item, within deciles defined globally in terms of PLM performance results. Cascading ordered probit and linear regression models were run with self-reports as the dependent variables and measured mobility, country, age, sex, education and obesity as independent variables. We examined correlations between self-reported items at the level of populations grouped into measured test deciles to assess patterns of response incomparability across items.

Results: For fixed levels of mobility, self-reports vary across countries, and the pattern of this variation differs between items. Independently of true mobility, self-reported mobility is influenced by a respondent's country, age, sex, and years of education. However, measured mobility levels and these individual characteristics explain only around 20% of the total variance in self-reports.

Conclusions: Our findings add to the growing body of literature on survey response incomparability across countries, groups within countries and individuals. We demonstrate that patterns of differential item functioning may not be detectable through traditional methods such as those based on item response theory. Highlighting determinants of self-report that are independent of measured health status points to future directions for improving comparability in population surveys. Our findings underscore the importance of further understanding individual characteristics that influence self-reporting on health.

Keywords: Health, Measurement, Mobility, Domains, Measured Tests, Self-Report