header

Searching Web Data: An Entity Retrieval and High-Performance Indexing Model

65 Pages Posted: 6 Jul 2018 Publication Status: Accepted

See all articles by Renaud Delbru

Renaud Delbru

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI)

Stephane Campinas

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI)

Giovanni Tummarello

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI)

Abstract

More and more (semi) structured information is becoming available on the Web in the form of documents embedding metadata (e.g., RDF, RDFa, Microformats and others). There are already hundreds of millions of such documents accessible and their number is growing rapidly. This calls for large scale systems providing effective means of searching and retrieving this semi-structured information with the ultimate goal of making it exploitable by humans and machines alike. This article examines the shift from the traditional web document model to a web data object (entity) model and studies the challenges faced in implementing a scalable and high performance system for searching semi-structured data objects over a large heterogeneous and decentralised infrastructure.Towards this goal, we define an entity retrieval model, develop novel methodologies for supporting this model and show how to achieve a high-performance entity retrieval system. We introduce an indexing methodology for semi-structured data which offers a good compromise between query expressiveness, query processing and index maintenance compared to other approaches. We address high-performance by optimisation of the index data structure using appropriate compression techniques. Finally, we demonstrate that the resulting system can index billions of data objects and provides keyword-based as well as more advanced search interfaces for retrieving relevant data objects in sub-second time. This work has been part of the Sindice search engine project at the Digital Enterprise Research Institute (DERI), NUI Galway. The Sindice system currently maintains more than 120 million pages downloaded from the Web and is being used actively by many researchers within and outside of DERI.

Keywords: Sindice, Information Retrieval, Entity Retrieval, Indexing Scheme, Compression, Semantic Web, Web Data, Semi-Structured Data, RDF, Inverted Index, Search Engine, Entity Search

Suggested Citation

Delbru, Renaud and Campinas, Stephane and Tummarello, Giovanni, Searching Web Data: An Entity Retrieval and High-Performance Indexing Model (2012). Journal of Web Semantics First Look, Available at SSRN: https://ssrn.com/abstract=3198931 or http://dx.doi.org/10.2139/ssrn.3198931

Renaud Delbru (Contact Author)

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI) ( email )

University Road
Galway, Co. Kildare
Ireland

Stephane Campinas

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI) ( email )

University Road
Galway, Co. Kildare
Ireland

Giovanni Tummarello

National University of Ireland, Galway (NUIG) - Digital Enterprise Research Institute (DERI) ( email )

University Road
Galway, Co. Kildare
Ireland

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
341
Downloads
17
PlumX Metrics