Learning to Detect Change: An Experimental Investigation

Abstract

People, across a wide range of personal and professional domains, need to accurately detect whether the state of the world has changed. Previous research has documented a systematic pattern of over- and under-reaction to signals of change due to system neglect, the tendency to overweight signals and underweight the system producing the signals. We investigate whether experience, and hence the potential to learn, improves people’s ability to detect change. Participants in our study made probabilistic judgments across 20 trials, each consisting of 10 periods, all in a single system that crossed three levels of diagnosticity (a measure of the informativeness of the signal) with four levels of transition probability (a measure of the stability of the environment). We found that the system-neglect pattern was only modestly attenuated by experience. Although average performance did not increase with experience overall, the degree of learning varied substantially across the 12 systems we investigated, with participants showing significant improvement in some high diagnosticity conditions and none in others. We examine this variation in learning through the lens of a simple linear adjustment heuristic, which we term the “δ-ϵ” model. We show that some systems produce consistent feedback in the sense that the best δ and ϵ responses for one trial also do well on other trials. We show that learning is related to the consistency of feedback, as well as a participant’s “scope for learning”—how close their initial judgments are to optimal behavior.

Keywords: change-point detection, regime shifts, learning, overreaction, underreaction, Bayesian updating, probability judgment

JEL Classification: C91, D83, D84, D91