Behavioral Feedback: Do Individual Choices Influence Scientific Results?

Abstract

In many health domains, we are concerned that observed links - for example, between “healthy” behaviors and good outcomes - are driven by selection into behavior. This paper considers the additional factor that these selection patterns may vary over time. When a particular health behavior becomes more recommended, the take-up of the behavior may be larger among people with other positive health behaviors. Such changes in selection would make it even more difficult to learn about causal effects. I formalize this change in selection in a simple model. I test for evidence of these patterns in the context of diet and vitamin supplementation. Using both microdata and evidence from published results I show that selection varies over time with recommendations about behavior and that estimates of the relationship between health outcomes and health behaviors vary over time in the same way. I show that adjustment for selection on observables is insufficient to address the bias. I suggest a possible robustness approach relying on assumptions about proportional selection of observed and unobserved variables.