Systematic DCG Evaluations of Search Algorithm Components

Posted: 30 Jan 2020

Date Written: January 29, 2020

Abstract

Many of the search algorithms used in LexisNexis applications are composed of individual components that each contribute to the relevance scores assigned to the members of result sets. In Lexis Advance, there are several such components called Levers, which can be enabled or disabled via query plans to alter the behavior of the search engine. Thirteen Levers are currently enabled in the Production search engine. These Levers were largely implemented and deployed before the current era of search accuracy testing using the STF tool. To determine how much each Lever individually contributes to the overall accuracy of the search engine, a series of experiments were conducted in STF in which the results of turning off each lever were compared with the system as it is configured now. From these experiments, we were able to determine which Levers had the most impact (whether positive or negative) and thus we able to target future efforts in improving the overall search algorithm. One of the major outcomes of this effort was obtaining the query-document pairs that needed to be rated by human testers in order to improve the coverage of tests. An overview of the testing apparatus is provided so that other organizations within the RELX Group may conduct similar assessments.

Keywords: search, relevancy, accuracy, metrics, algorithm, hrt, dcg, hdcg, evaluation

Suggested Citation

Bignoli, Perrin, Systematic DCG Evaluations of Search Algorithm Components (January 29, 2020). Proceedings of the 3rd Annual RELX Search Summit, Available at SSRN: https://ssrn.com/abstract=3527352

Perrin Bignoli (Contact Author)

LexisNexis ( email )

P. O. Box 933
Dayton, OH 45401
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
116
PlumX Metrics