A Sample Size Calculation for Training and Certifying Targeting Policies

67 Pages Posted: 6 Oct 2022 Last revised: 29 Jun 2023

See all articles by Duncan Simester

Duncan Simester

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Artem Timoshenko

Kellogg School of Management, Northwestern University

Spyros I. Zoumpoulis

INSEAD - Decision Sciences

Date Written: September 22, 2022

Abstract

We propose an approach for determining the sample size required when using an experiment to train and certify a targeting policy. Calculating the rate at which the performance of a targeting model improves with additional training data is a complex problem. We address this challenge by assuming that customers are grouped into segments that capture relevant information about their responsiveness to the firm's marketing actions. We consider two problem formulations. The first formulation identifies the sample size required to train a targeting policy and certify that its expected performance exceeds a predefined threshold. The second formulation identifies the sample size required to train a targeting policy and certify that it outperforms a baseline in an out-of-sample statistical test. We establish theoretical properties of these problems, based on which we propose computationally efficient algorithms for optimal sample size calculations. We illustrate our algorithms and analysis using data from a luxury fashion retailer.

Keywords: sample size calculation, targeting policies, policy training, policy certification, experimentation

Suggested Citation

Simester, Duncan and Timoshenko, Artem and Zoumpoulis, Spyros, A Sample Size Calculation for Training and Certifying Targeting Policies (September 22, 2022). MIT Sloan Research Paper No. 6802-22, INSEAD Working Paper No. 2024/48/DSC, Available at SSRN: https://ssrn.com/abstract=4228297 or http://dx.doi.org/10.2139/ssrn.4228297

Duncan Simester

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

Management Science
Cambridge, MA 02142
United States
617-258-0679 (Phone)
617-258-7597 (Fax)

Artem Timoshenko (Contact Author)

Kellogg School of Management, Northwestern University ( email )

2001 Sheridan Road
Evanston, IL 60208
United States

Spyros Zoumpoulis

INSEAD - Decision Sciences ( email )

France

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
270
Abstract Views
1,512
Rank
232,419
PlumX Metrics