Studies in the Evaluation of a Domain-Independent Natural Language Query System

Abstract

There is growing consensus that some of the most crucial questionsconcerning the feasibility and desirability of natural language interfaces todatabases can only be resolved by empirical research. This paper reports theresults of several empirical studies which investigated the samedomain-independent natural language query system, using various applications intwo different natural languages - English and German. Taken together, theseexperiments involved about 100 subjects and over 12,000 queries, constitutingthe bulk of empirical evaluations of natural query language systems reported todate. Some definitive results are derived from the combined experience, andplans are outlined to resolve several of the remaining issues.