Optimization in Online Content Recommendation Services: Beyond Click-Through Rates

Manufacturing & Service Operations Management. 2016, Vol. 18 (1), Pages 15-33.

Columbia Business School Research Paper No. 14-33

19 Pages Posted: 1 Aug 2014 Last revised: 2 Feb 2016

See all articles by Omar Besbes

Omar Besbes

Columbia Business School - Decision Risk and Operations

Yonatan Gur

Stanford Graduate School of Business

Assaf Zeevi

Columbia Business School - Decision Risk and Operations

Date Written: March 26, 2015

Abstract

A new class of online services allows internet media sites to direct users from articles they are currently reading to other content they may be interested in. This process creates a "browsing path'' along which there is potential for repeated interaction between the user and the provider, giving rise to a dynamic optimization problem. A key metric that often underlies this recommendation process is the click-through rate (CTR) of candidate articles. While CTR is a measure of instantaneous click likelihood, we analyze the performance improvement that one may achieve by some lookahead that accounts for the potential future path of users. To that end, using a large data set of user path history at major media sites, we introduce and derive a representation of content along two key dimensions: clickability, the likelihood to click to an article when it is recommended; and engageability, the likelihood to click from an article when it hosts a recommendation. We then propose a class of heuristics that leverage both clickability and engageability, and provide theoretical support for favoring such path-focused heuristics over myopic heuristics that focus only on clickability (no lookahead). We conduct a live pilot experiment that measures the performance of a practical proxy of our proposed class, when integrated into the operating system of a worldwide leading provider of content recommendations. We estimate the aggregate improvement in clicks-per-visit relative to the CTR-driven current practice. The documented improvement highlights the importance and the practicality of efficiently incorporating for the future path of users in real time.

Keywords: online services, dynamic assortment selection, data-driven optimization, recommendation systems, content marketing, digital marketing, path data.

Suggested Citation

Besbes, Omar and Gur, Yonatan and Zeevi, Assaf, Optimization in Online Content Recommendation Services: Beyond Click-Through Rates (March 26, 2015). Manufacturing & Service Operations Management. 2016, Vol. 18 (1), Pages 15-33. ; Columbia Business School Research Paper No. 14-33. Available at SSRN: https://ssrn.com/abstract=2474049 or http://dx.doi.org/10.2139/ssrn.2474049

Omar Besbes

Columbia Business School - Decision Risk and Operations ( email )

New York, NY
United States

Yonatan Gur (Contact Author)

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Assaf Zeevi

Columbia Business School - Decision Risk and Operations ( email )

New York, NY
United States
212-854-9678 (Phone)
212-316-9180 (Fax)

HOME PAGE: http://www.gsb.columbia.edu/faculty/azeevi/

Register to save articles to
your library

Register

Paper statistics

Downloads
631
Abstract Views
2,458
rank
41,445
PlumX Metrics