header

A Benchmark for End-User Structured Data User Interfaces

29 Pages Posted: 20 Jan 2021 Publication Status: Accepted

See all articles by Roberto García

Roberto García

University of Lleida - Departament d'Informàtica i Enginyeria Industrial

Rosa Gil

University of Lleida - Departament d'Informàtica i Enginyeria Industrial

Eirik Bakke

Computer Science and Artificial Intelligence Laboratory, MIT

David Karger

Massachusetts Institute of Technology (MIT) - Computer Science and Artificial Intelligence Laboratory (CSAIL)

Abstract

During the years, it has been possible to assess significant improvements in the computational efficiency of Semantic Web search and exploration systems. However, it has been much harder to assess how well different semantic systems' user interfaces help their users. One of the key factors facilitating the advancement of research in a particular field is the ability to compare the performance of different approaches. Though there are many such benchmarks in Semantic Web fields that have experienced significant improvements, this is not the case for Semantic Web user interfaces for data exploration. We propose and demonstrate the use of a benchmark for evaluating such user interfaces, which includes a set of typical user tasks and a well-defined procedure for assigning a measure of performance on those tasks to a semantic system. We have applied the benchmark to four such systems. Moreover, all the required resources to apply the benchmark are openly available online. We intend to initiate a community conversation that will lead to a generally accepted framework for comparing systems and for measuring, and thus encouraging, progress towards better semantic search and exploration tools.

Keywords: benchmark, user experience, usability, semantic data, exploration, relational data

Suggested Citation

García, Roberto and Gil, Rosa and Bakke, Eirik and Karger, David, A Benchmark for End-User Structured Data User Interfaces. Journal of Web Semantics First Look , Available at SSRN: https://ssrn.com/abstract=3769925 or http://dx.doi.org/10.2139/ssrn.3769925

Roberto García (Contact Author)

University of Lleida - Departament d'Informàtica i Enginyeria Industrial ( email )

C / Jaume II, 69
Lleida, 25001
Spain

Rosa Gil

University of Lleida - Departament d'Informàtica i Enginyeria Industrial ( email )

C / Jaume II, 69
Lleida, 25001
Spain

Eirik Bakke

Computer Science and Artificial Intelligence Laboratory, MIT ( email )

David Karger

Massachusetts Institute of Technology (MIT) - Computer Science and Artificial Intelligence Laboratory (CSAIL) ( email )

Stata Center
Cambridge, MA 02142
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
74
Downloads
3
PlumX Metrics