Download this Paper Open PDF in Browser

Comparison of Approaches for Gathering Data from the Web for Technology Trend Analysis

27 Pages Posted: 11 Mar 2009  

Ayse Kaya Firat

Massachusetts Institute of Technology (MIT) - Department of Electrical Engineering and Computer Science

Wei Lee Woon

Masdar Institute of Science and Technology (MIST)

Stuart Madnick

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Date Written: August 1, 2008

Abstract

We are investigating trend extrapolation using historical data from academic publications to forecast
future technology directions. Many sources of academic information on the Web (e.g., Google Scholar,
Scirus) provide a wealth of relevant information, yet they are not structured for programmatic access. We
can use Web wrappers, programs that can harvest text data from Web pages and present them in a
structured format, to overcome this problem.

JEL Classification: trend analysis, web wrappers, google scholar

Suggested Citation

Kaya Firat, Ayse and Woon, Wei Lee and Madnick, Stuart, Comparison of Approaches for Gathering Data from the Web for Technology Trend Analysis (August 1, 2008). MIT Sloan Research Paper No. 4727-09. Available at SSRN: https://ssrn.com/abstract=1356047 or http://dx.doi.org/10.2139/ssrn.1356047

Ayse Kaya Firat (Contact Author)

Massachusetts Institute of Technology (MIT) - Department of Electrical Engineering and Computer Science ( email )

77 Massachusetts Ave
Cambridge, MA 02139
United States

Wei Lee Woon

Masdar Institute of Science and Technology (MIST) ( email )

MASDAR
PO Box 54115
Abu Dhabi
United Arab Emirates

Stuart Madnick

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

E53-321
Cambridge, MA 02142
United States
617-253-6671 (Phone)
617-253-3321 (Fax)

Paper statistics

Downloads
149
Rank
169,882
Abstract Views
6,764