Deep-Learning-Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence

71 Pages Posted: 14 Mar 2023 Last revised: 25 Jan 2024

See all articles by Zikun Ye

Zikun Ye

University of Washington - Michael G. Foster School of Business

Zhiqi Zhang

Washington University in St. Louis

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School

Heng Zhang

Supply Chain Management Department - W.P.Carey School of Business

Renyu (Philip) Zhang

The Chinese University of Hong Kong

Date Written: March 1, 2023

Abstract

Large-scale online platforms launch hundreds of randomized experiments (a.k.a. A/B tests) every day to iterate their operations and marketing strategies, while the combinations of these treatments are typically not exhaustively tested. It triggers an important question of both academic and practical interests: Without observing the outcomes of all treatment combinations, how to estimate the causal effect of any treatment combination and identify the optimal treatment combination? We develop a novel framework combining deep learning and doubly robust estimation to estimate the causal effect of any treatment combination for each user on the platform when observing only a small subset of treatment combinations. Our proposed framework (called debiased deep learning, DeDL) exploits Neyman orthogonality and combines interpretable and flexible structural layers in deep learning. We prove theoretically that this framework yields efficient, consistent, and asymptotically normal estimators under mild assumptions, thus allowing for identifying the best treatment combination when only observing a few combinations. To empirically validate our method, we then collaborate with a large-scale video-sharing platform and implement our framework for three experiments involving three treatments where each combination of treatments is tested. When only observing a subset of treatment combinations, our DeDL approach significantly outperforms other benchmarks to accurately estimate and infer the average treatment effect (ATE) of any treatment combination, and to identify the optimal treatment combination.

Keywords: Deep Learning, Double Machine Learning, Causal Inference, Field Experiments, Experimentation on Online Platforms

Suggested Citation

Ye, Zikun and Zhang, Zhiqi and Zhang, Dennis and Zhang, Heng and Zhang, Renyu, Deep-Learning-Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence (March 1, 2023). Available at SSRN: https://ssrn.com/abstract=4375327 or http://dx.doi.org/10.2139/ssrn.4375327

Zikun Ye (Contact Author)

University of Washington - Michael G. Foster School of Business ( email )

Seattle, WA 98195
United States

Zhiqi Zhang

Washington University in St. Louis ( email )

St. Louis, MO
United States

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Heng Zhang

Supply Chain Management Department - W.P.Carey School of Business ( email )

Tempe, AZ
United States

Renyu Zhang

The Chinese University of Hong Kong ( email )

Shatin, N.T.
Hong Kong, Hong Kong
China

HOME PAGE: http://rphilipzhang.github.io/rphilipzhang/index.html

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
678
Abstract Views
2,369
Rank
72,863
PlumX Metrics