Non-stationary Experimental Design under Structured Trends

54 Pages Posted: 25 Jul 2023 Last revised: 20 Oct 2023

See all articles by David Simchi-Levi

David Simchi-Levi

Massachusetts Institute of Technology (MIT) - School of Engineering

Chonghuan Wang

Massachusetts Institute of Technology (MIT)

Zeyu Zheng

University of California, Berkeley

Date Written: July 18, 2023

Abstract

Experimentation has become increasingly popular across various domains, such as clinical trials and online platforms, due to its widely recognized benefits. One of the primary objectives of classical experiments is to estimate the average treatment effect (ATE) to inform future decision-making. However, in healthcare and many other settings, treatment effects may be non-stationary, meaning that they can change over time, rendering the traditional experimental design inadequate and the classical static ATE uninformative. In this work, we address the problem of non-stationary experimental design under structured trends by considering two objectives: estimating the dynamic treatment effect and minimizing welfare loss within the experiment. We propose an efficient design that can be customized for optimal estimation error rate, optimal regret rate, or the Pareto optimal trade-off between the two objectives. We establish information-theoretical lower bounds that highlight the inherent challenge in estimating dynamic treatment effects and minimizing welfare loss, and also statistically reveal the fundamental trade-off between the two objectives.

Keywords: Adaptive Experimental Design, Non-stationary, Online Learning, Treatment Effect

Suggested Citation

Simchi-Levi, David and Wang, Chonghuan and Zheng, Zeyu, Non-stationary Experimental Design under Structured Trends (July 18, 2023). Available at SSRN: https://ssrn.com/abstract=4514568 or http://dx.doi.org/10.2139/ssrn.4514568

David Simchi-Levi

Massachusetts Institute of Technology (MIT) - School of Engineering ( email )

MA
United States

Chonghuan Wang (Contact Author)

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

Zeyu Zheng

University of California, Berkeley ( email )

4125 Etcheverry Hall
Berkeley, CA 94720
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
217
Abstract Views
450
Rank
249,698
PlumX Metrics