header

Ontology Matching Benchmarks: Generation, Stability, and Discriminability

25 Pages Posted: 7 Jul 2018 Publication Status: Accepted

See all articles by Jérôme Euzenat

Jérôme Euzenat

Université de Nice Sophia Antipolis - INRIA - Institut National de Recherche en Informatique et Automatique; Laboratoire d'Informatique de Grenoble (LIG)

Maria-Elena Rosoiu

Université de Nice Sophia Antipolis - INRIA - Institut National de Recherche en Informatique et Automatique; Laboratoire d'Informatique de Grenoble (LIG)

Cássia Trojahn

Institut de Recherche en Informatique de Toulouse (IRIT)

Abstract

The OAEI Benchmark test set has been used for many years as a main reference to evaluate and compare ontology matching systems. However, this test set has barely varied since 2004 and has become a relatively easy task for matchers. In this paper, we present the design of a flexible test generator based on an extensible set of alterators which may be used programmatically for generating different test sets from different seed ontologies and different alteration modalities. It has been used for reproducing Benchmark both with the original seed ontology and with other ontologies. This highlights the remarkable stability of results over different generations and the preservation of difficulty across seed ontologies, as well as a systematic bias towards the initial Benchmark test set and the inability of such tests to identify an overall winning matcher. These were exactly the properties for which Benchmark had been designed. Furthermore, the generator has been used for providing new test sets aiming at increasing the difficulty and discriminability of Benchmark. Although difficulty may be easily increased with the generator, attempts to increase discriminability proved unfruitful. However, efforts towards this goal raise questions about the very nature of discriminability.

Keywords: Ontology Matching, Matching Evaluation, Test Generation, Semantic Web

Suggested Citation

Euzenat, Jérôme and Rosoiu, Maria-Elena and Trojahn, Cássia, Ontology Matching Benchmarks: Generation, Stability, and Discriminability (August 2013). Journal of Web Semantics First Look, Available at SSRN: https://ssrn.com/abstract=3199066 or http://dx.doi.org/10.2139/ssrn.3199066

Jérôme Euzenat (Contact Author)

Université de Nice Sophia Antipolis - INRIA - Institut National de Recherche en Informatique et Automatique ( email )

250, rue Albert Einstein
Sophia Antipolis
France

Laboratoire d'Informatique de Grenoble (LIG) ( email )

Bâtiment IMAG, 700 avenue Centrale
Saint-Martin-de-Pallières
France

Maria-Elena Rosoiu

Université de Nice Sophia Antipolis - INRIA - Institut National de Recherche en Informatique et Automatique ( email )

250, rue Albert Einstein
Sophia Antipolis
France

Laboratoire d'Informatique de Grenoble (LIG) ( email )

Bâtiment IMAG, 700 avenue Centrale
Saint-Martin-de-Pallières
France

Cássia Trojahn

Institut de Recherche en Informatique de Toulouse (IRIT) ( email )

Toulouse
France

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
196
Downloads
13
PlumX Metrics