Web Scraping Housing Prices in Real-time: the Covid-19 Crisis in the UK

42 Pages Posted: 7 Sep 2021

Date Written: august 2021


While official statistics provide lagged and aggregate information on the housing market, extensive information is available publicly on real-estate websites. By web scraping them for the UK on a daily basis, this paper extracts a large database from which we build timelier and highly granular indicators. One originality of the dataset is to provide the sellers’ perspective, allowing to compute innovative indicators of the housing market such as the numb er of new posted offers or how prices fluctuate over time for existing offers. Matching selling prices in our dataset with transacted prices from the notarial database using machine learning techniques allows us to measure the negotiation margin of buyers – an innovation to the literature. During the Covid-19 crisis, these indicators demonstrate the freezing of the market and the “wait-and-see” behaviour of sellers. They also show that prices have been increasing in rural regions after the lockdown but experienced a continued decline in London.

Keywords: Housing, Real-time, Big Data, Web Scraping, High Frequency, United Kingdom

JEL Classification: E01, R30

Suggested Citation

Bricongne, Jean-Charles and Meunier, Baptiste and Sylvain, Pouget, Web Scraping Housing Prices in Real-time: the Covid-19 Crisis in the UK (august 2021). Banque de France Working Paper No. 827, Available at SSRN: https://ssrn.com/abstract=3916196 or http://dx.doi.org/10.2139/ssrn.3916196

Jean-Charles Bricongne (Contact Author)

Banque de France ( email )


Baptiste Meunier

Banque de France ( email )


Pouget Sylvain

Banque de France ( email )


Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics