Web Data Collection

Abstract

The digital revolution and the widespread use of the internet have had profound effects on people's everyday lives as well as empirical social science research. The digital landscape that has developed with this shift holds unprecedented masses of data which are increasingly used by sociologists. Popular sources of web data include for example data from social media sites, news sites, digitised archives, crowd-sourced information, or online markets. How can this data be harnessed? This chapter provides a hands-on introduction to collecting web data directly from platforms, providing code examples in two popular programming languages-R and Python.

Keywords: API, application programming interface, data collection, digital trace data, web data, web scraping