Insights from Implementing Click DCG in Adobe Analytics

Posted: 14 Nov 2019

Date Written: November 13, 2019

Abstract

Discounted Cumulative Gain (DCG) is a commonly used metric for evaluating the precision and ranking quality of search engines results. It provides a common baseline to compare and improve search algorithms in web applications, for example through A/B testing. While the metric is proven to be useful for product improvement of products with search functionality, it has not been implemented in common Web Analytics products, such as Adobe Analytics or Google Analytics.

In this work, we have implemented DCG in Adobe Analytics. This implementation allows computation of DCG@3, DCG@5, and DCG@25 using the standard log2 model. The solution uses the click-through rates for search results, hence we coin this version of DCG: Click DCG (cDCG). We show which events need to be triggered on the website and how an Adobe Analytics Calculated Metric can be built to compute cDCG.

Further to implementation, we demonstrate the value and rich insights we can gained from analyzing over 50 search pages at Elsevier, by breaking cDCG down by over 100 dimensions available in Adobe Analytics. For example, we show how cDCG can be used in query analysis to identify query terms that perform below average. And, we present how the users’ origin (i.e. country) affects cDCG. Lastly, we demonstrate how cDCG can be used in A/B experiments using Adobe Target, and how it was used in an actual A/B experiment to validate that cDCG values remained at high levels while migrating the search backend to a new technology.

Keywords: Web Analytics, Adobe Analytics, Search Optimization, DCG

Suggested Citation

Zazworka, Nico and van Acker, Saskia and Fukunari, Hiroshi and Tsaousi, Tatiana, Insights from Implementing Click DCG in Adobe Analytics (November 13, 2019). Proceedings of the 3rd Annual RELX Search Summit, Available at SSRN: https://ssrn.com/abstract=3486162

Saskia Van Acker

Elsevier Amsterdam ( email )

Radarweg 29
Amsterdam, 1043 NX
Netherlands

Hiroshi Fukunari

Elsevier ( email )

Tatiana Tsaousi

Elsevier ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
392
PlumX Metrics