A Sample Size Calculation for Training and Certifying Targeting Policies
67 Pages Posted: 6 Oct 2022 Last revised: 29 Jun 2023
Date Written: September 22, 2022
Abstract
We propose an approach for determining the sample size required when using an experiment to train and certify a targeting policy. Calculating the rate at which the performance of a targeting model improves with additional training data is a complex problem. We address this challenge by assuming that customers are grouped into segments that capture relevant information about their responsiveness to the firm's marketing actions. We consider two problem formulations. The first formulation identifies the sample size required to train a targeting policy and certify that its expected performance exceeds a predefined threshold. The second formulation identifies the sample size required to train a targeting policy and certify that it outperforms a baseline in an out-of-sample statistical test. We establish theoretical properties of these problems, based on which we propose computationally efficient algorithms for optimal sample size calculations. We illustrate our algorithms and analysis using data from a luxury fashion retailer.
Keywords: sample size calculation, targeting policies, policy training, policy certification, experimentation
Suggested Citation: Suggested Citation