The Market Basket Transformer: A New Foundation Model for Retail

Abstract

This paper proposes a market basket transformer (MBT), an adaptation of large language model architectures to market basket data. Our transformer learns purchase patterns in unordered product sets, predicting basket composition more than twice as accurately as existing approaches. Three features differentiate the MBT from large language models. First, we incorporate covariates to account for varying purchase probabilities across heterogeneous shopping trips. Second, we improve the pretraining of the transformer by leveraging available data more efficiently. Third, we generate synthetic training examples to ensure that the market basket transformer can learn meaningful product representations for all products in the retailer's assortment. These extensions ensure that the MBT accurately learns purchase patterns, especially for long-tail products typically underrepresented in the training data. We demonstrate the model’s adaptability by fine-tuning it to predict coupon redemptions using data from a coupon experiment. Because the market basket transformer learns rich patterns in market basket data, it is a foundation model that can be easily adapted to various retail analytics applications.

Keywords: transformer models, foundation models, fine-tuning, retailing analytics, market basket analysis, long-tail products