Personalized Recommendation System Design for an Online B2B Platform
46 Pages Posted: 12 Aug 2021 Last revised: 23 Jan 2024
Date Written: November 11, 2020
Abstract
We solve the problem of designing a personalized recommendation system for an online business-to-business (B2B) marketplace and evaluate its performance using out-of-sample test data and field experiments. In this problem, buyers place requests for quotation (RFQs) to the platform, sellers respond by accepting or declining those RFQs, and the objective of the recommender is to generate a real-time ranked list of relevant RFQs matched to each seller. The key technical challenge in this setting is of class imbalance such that the volume of `accepted' records is significantly larger than `declined' records. We construct a recommender using an attribute-based logit classifier, represent RFQs using calibrated features for the short timespan of RFQs and high-dimensional and sparse transaction data, and conduct synthetic data augmentation using a resampling approach we propose, Panel Data Augmentation Technique (PDATE), to mitigate class imbalance. By leveraging proprietary data of a dominant online B2B platform in India, we demonstrate the existence of class imbalance, conduct an out-of-sample evaluation of PDATE and compare its performance against the standard method of Synthetic Minority Oversampling Technique (SMOTE), and propose a novel aggregate measure based on operational costs to determine the optimal resampling strategy. The true value of a recommendation system with latent consideration sets can only be determined in live implementation. Thus, we conduct two controlled field experiments at the platform and show that our method leads to significant improvements in the top-5, top-10, and top-25 contributions from the recommendations page. Our method has been adopted by the platform for all their customers, and demonstrates the value of algorithmic recommendation system design for an online B2B platform.
Keywords: Online marketplace, recommendation system, high-dimensionality, class imbalance, choice estimation, field experiment
JEL Classification: C51, C52, C55, C93, L81, M11, M21
Suggested Citation: Suggested Citation