Does Human-Algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow's Zestimate

44 Pages Posted: 29 Mar 2022 Last revised: 21 Jun 2023

See all articles by Runshan Fu

Runshan Fu

New York University (NYU) - Leonard N. Stern School of Business

Ginger Zhe Jin

University of Maryland - Department of Economics; National Bureau of Economic Research (NBER)

Meng Liu

Washington University in St. Louis

Date Written: March 2022

Abstract

We study how home sellers and buyers interact with Zillow’s Zestimate algorithm throughout the sales cycle of residential properties, with an emphasis on the implications of such interactions. In particular, leveraging Zestimate’s algorithm updates as exogenous shocks, we find evidence for a human-algorithm feedback loop: listing and selling outcomes respond significantly to Zestimate, and Zestimate is quickly updated for the focal and comparable homes after a property is listed or sold. This raises a concern that housing market disturbances may propagate and persist because of the feedback loop. However, simulation suggests that disturbances are short-lived and diminish eventually, mainly because all marginal effects across stages of the selling process—though sizable and significant—are less than one. To further validate this insight in the real data, we leverage the COVID-19 pandemic as a natural experiment. We find consistent evidence that the initial disturbances created by the March-2020 declaration of national emergency faded away in a few months. Overall, our results identify the human-algorithm feedback loop in an important real-world setting, but dismiss the concern that such a feedback loop generates persistent error propagation.

Suggested Citation

Fu, Runshan and Jin, Ginger Zhe and Liu, Meng, Does Human-Algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow's Zestimate (March 2022). NBER Working Paper No. w29880, Available at SSRN: https://ssrn.com/abstract=4068062

Runshan Fu (Contact Author)

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

Ginger Zhe Jin

University of Maryland - Department of Economics ( email )

College Park, MD 20742
United States
301-405-3484 (Phone)
301-405-3542 (Fax)

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Meng Liu

Washington University in St. Louis ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
80
Abstract Views
350
Rank
290,020
PlumX Metrics