Fusing Visual and Mobility Data for City Sensing: A Case Study of Urban Village Recognition

26 Pages Posted: 17 Jan 2023

See all articles by Yingjing Huang

Yingjing Huang

Peking University

Fan Zhang

Hong Kong University of Science and Technology

Yong Gao

Peking University

Wei TU

Shenzhen University

Fábio Duarte

Massachusetts Institute of Technology (MIT)

Carlo Ratti

Massachusetts Institute of Technology (MIT)

Diansheng Guo

affiliation not provided to SSRN

Yu Liu

Peking University - Institute of Remote Sensing and Geographical Information Systems

Abstract

Using multimodal data fusion for city sensing can lead to more accurate, comprehensive and reliable results. Existing studies have used the strategy of fusion of remote sensing and social sensing to make the results least resource-intensive and most accurate. However, it is difficult to distinguish some similar urban functions using only remote sensing to represent visual information in spatial units. To fill the gap, this paper designs Sensing Blender, an end-to-end deep learning model that integrates remote sensing, street view imagery and social sensing to comprehensively characterize urban space. Specifically, this model combines physical environment features from satellite imagery and street view imagery with the dynamic mobility features from taxi trajectory data. In particular, a novel module in Sensing Blender is proposed to extract features from varying numbers of street view images. To validate the performance of the proposed model, a series of experiments of urban village recognition were conducted in Shenzhen, China, with a grid resolution of 500 meters. The results indicate that Sensing Blender achieved good performance with an overall accuracy (OA) of 92.0% and the Kappa of 0.720. Compared with unimodal models, our multimodal model improved the OA by 9.2% and the Kappa by 0.179. The proposed model provides an effective and efficient method for monitoring the distribution of urban villages, potentially supporting urban management, decision-making, and research on urban expansion and urban renewal.

Keywords: City Sensing, multimodal data fusion, Remote sensing, social sensing, deep learning, urban village recognition

Suggested Citation

Huang, Yingjing and Zhang, Fan and Gao, Yong and TU, Wei and Duarte, Fábio and Ratti, Carlo and Guo, Diansheng and Liu, Yu, Fusing Visual and Mobility Data for City Sensing: A Case Study of Urban Village Recognition. Available at SSRN: https://ssrn.com/abstract=4326601 or http://dx.doi.org/10.2139/ssrn.4326601

Yingjing Huang

Peking University ( email )

No. 38 Xueyuan Road
Haidian District
Beijing, 100871
China

Fan Zhang (Contact Author)

Hong Kong University of Science and Technology ( email )

Hong Kong

Yong Gao

Peking University ( email )

No. 38 Xueyuan Road
Haidian District
Beijing, 100871
China

Wei TU

Shenzhen University ( email )

3688 Nanhai Road, Nanshan District
Shenzhen, 518060
China

Fábio Duarte

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

Carlo Ratti

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

Diansheng Guo

affiliation not provided to SSRN ( email )

No Address Available

Yu Liu

Peking University - Institute of Remote Sensing and Geographical Information Systems ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
145
Abstract Views
467
Rank
428,598
PlumX Metrics