Does Human-algorithm Feedback Loop Lead To Error Propagation? Evidence from Zillow’s Zestimate

Abstract

We study how home sellers and buyers interact with Zillow's Zestimate algorithm throughout the sales cycle of residential properties, with an emphasis on the implications of such interactions. In particular, leveraging Zestimate's algorithm updates as exogenous shocks, we find evidence for a human-algorithm feedback loop: listing and selling outcomes respond significantly to Zestimate, and Zestimate is quickly updated for the focal and comparable homes after a property is listed or sold. This raises a concern that housing market disturbances may propagate and persist because of the feedback loop. However, simulation suggests that disturbances are short-lived and diminish eventually, mainly because all marginal effects across stages of the selling process---though sizable and significant---are less than one. To further validate this insight in the real data, we leverage the COVID-19 pandemic as a natural experiment. We find consistent evidence that the initial disturbances created by the March-2020 declaration of national emergency faded away in a few months. Overall, our results identify the human-algorithm feedback loop in an important real-world setting, but dismiss the concern that such a feedback loop generates persistent error propagation.

Keywords: Algorithm, Human-algorithm feedback loop, Error Propagation, Zillow, Zestimate, Artificial Intelligence

JEL Classification: D83, L11, L85, L86