Correlated Cluster-Based Randomized Experiments: Robust Variance Minimization

Forthcoming, Management Science. https://doi.org/10.1287/mnsc.2021.02741

Chicago Booth Research Paper No. 21-17.

66 Pages Posted: 1 Jun 2021 Last revised: 9 Aug 2023

See all articles by Ozan Candogan

Ozan Candogan

University of Chicago - Booth School of Business

Chen Chen

New York University (NYU) - New York University (NYU), Shanghai

Rad Niazadeh

University of Chicago - Booth School of Business

Date Written: May 24, 2021

Abstract

Experimentation is prevalent in online marketplaces and social networks to assess the effectiveness of new market intervention. To mitigate the interference among users in an experiment, a common practice is to use a cluster-based experiment, where the designer partitions the market into loosely connected clusters and assigns all users in the same cluster to the same variant (treatment or control). Given the experiment, we assume an unbiased Horvitz-Thompson estimator is used to estimate the total market effect of the treatment. We consider the optimization problem of choosing (correlated) randomized assignments of clusters to treatment and control to minimize the worst-case variance of the estimator under a constraint that the marginal assignment probability is q \in (0,1) for all clusters. This problem can be formulated as a linear program where both the number of decision variables and constraints are exponential in the number of clusters---and hence is generally computationally intractable.

We develop a family of practical experiments that we refer to as \emph{independent block randomization (IBR)} experiments. Such an experiment partitions clusters into blocks so that each block contains clusters of similar size. It then treats a fraction q of the clusters in each block (chosen uniformly at random) and does so independently across blocks. The optimal cluster partition can be obtained in a tractable way using dynamic programming. We show that these policies are asymptotically optimal when the number of clusters grows large and no cluster size dominates the rest. In the special case where cluster sizes take values in a finite set and the number of clusters of each size is a fixed proportion of the total number of clusters, the loss is only a constant that is independent of the number of clusters. Beyond the asymptotic regime, we show that the IBR experiment has a good approximation for any problem instance when q is not very tiny. We also examine the performance of the IBR experiments on data-driven numerical examples, including examples based on Airbnb and Facebook data.

Keywords: Variance minimization, robust optimization, experimental design, cluster-based randomization, approximation algorithms, asymptotic optimality.

Suggested Citation

Candogan, Ozan and Chen, Chen and Niazadeh, Rad, Correlated Cluster-Based Randomized Experiments: Robust Variance Minimization (May 24, 2021). Forthcoming, Management Science. https://doi.org/10.1287/mnsc.2021.02741, Chicago Booth Research Paper No. 21-17., Available at SSRN: https://ssrn.com/abstract=3852100 or http://dx.doi.org/10.2139/ssrn.3852100

Ozan Candogan

University of Chicago - Booth School of Business ( email )

5807 S. Woodlawn Avenue
Chicago, IL 60637
United States

HOME PAGE: http://faculty.chicagobooth.edu/ozan.candogan/

Chen Chen (Contact Author)

New York University (NYU) - New York University (NYU), Shanghai ( email )

567 West Yangsi Rd
Shanghai, Shanghai 200124
China

Rad Niazadeh

University of Chicago - Booth School of Business ( email )

5807 S Woodlawn Ave
Chicago, IL 60637

HOME PAGE: http://https://faculty.chicagobooth.edu/rad-niazadeh

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
583
Abstract Views
2,988
Rank
78,830
PlumX Metrics