header

WebPIE: A Web-Scale Parallel Inference Engine Using MapReduce

46 Pages Posted: 6 Jul 2018 Publication Status: Accepted

See all articles by Jacopo Urbani

Jacopo Urbani

VU University Amsterdam - Department of Computer Science

Spyros Kotoulas

VU University Amsterdam

Jason Massen

VU University Amsterdam - Department of Computer Science

Frank van Harmelen

VU University Amsterdam - Department of Artificial Intelligence

Henri Bal

VU University Amsterdam - Department of Computer Science

Abstract

The large amount of Semantic Web data and its fast growth pose a significant computational challenge in performing efficient and scalable reasoning. On a large scale, the resources of single machines are no longer sufficient and we are required to distribute the process to improve performance.  In this article, we propose a distributed technique to perform materialization under the RDFS and OWL ter Horst semantics using the MapReduce programming model. We will show that a straightforward implementation is not efficient and does not scale. Our technique addresses the challenge of distributed reasoning through a set of algorithms which, combined, significantly increase performance.  We have implemented WebPIE (Web-scale Inference Engine) and we demonstrate its performance on a cluster of up to 64 nodes. We have evaluated our system using very large real-world datasets (Bio2RDF, LLD, FactForge and the Billion Triple Challenge dataset) and the LUBM synthetic benchmark, scaling up to 100 billion triples. Results show that our implementation scales linearly and vastly outperforms current systems in terms of maximum data size and inference speed.

Keywords: Semantic Web, MapReduce, high performance, distributed computing, reasoning

Suggested Citation

Urbani, Jacopo and Kotoulas, Spyros and Massen, Jason and van Harmelen, Frank and Bal, Henri, WebPIE: A Web-Scale Parallel Inference Engine Using MapReduce (2012). Journal of Web Semantics First Look, Available at SSRN: https://ssrn.com/abstract=3198932 or http://dx.doi.org/10.2139/ssrn.3198932

Jacopo Urbani (Contact Author)

VU University Amsterdam - Department of Computer Science ( email )

De Boelelaan 1081
1081 HV Amsterdam
Netherlands

Spyros Kotoulas

VU University Amsterdam ( email )

De Boelelaan 1105
Amsterdam, ND North Holland 1081 HV
Netherlands

Jason Massen

VU University Amsterdam - Department of Computer Science ( email )

De Boelelaan 1081
1081 HV Amsterdam
Netherlands

Frank Van Harmelen

VU University Amsterdam - Department of Artificial Intelligence ( email )

De Boelelaan 1081
1081 HV Amsterdam
Netherlands

Henri Bal

VU University Amsterdam - Department of Computer Science ( email )

De Boelelaan 1081
1081 HV Amsterdam
Netherlands

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
283
Downloads
30
PlumX Metrics