Informing Policy with Text Mining: Technological Change and Social Challenges
47 Pages Posted: 2 May 2019
Date Written: March 29, 2019
The fast development and adoption of ICT technologies and digital services has major social implications. Policy-makers often struggle to design appropriate regulations, defend the rights of citizens and ensure competition. The aim of this work is to present various methodologies that support stakeholders with identification of emerging technologies and related social challenges based on the text mining of news articles. The analysis demonstrates that while each text mining algorithm provides insightful results, their combination yields more detailed and robust overview of media discussions. The results present early signals and trends, the relationships between technologies and social challenges, and changing attitudes towards selected tech issues.
Keywords: online news, web scraping, text mining, sentiment analysis, topic-modelling
JEL Classification: 033, 038
Suggested Citation: Suggested Citation