Learning, Private Information, and the Economic Evaluation of Randomized Experiments

Olin School of Business Working Paper No. 2003-08-03

52 Pages Posted: 15 Oct 2003

See all articles by Barton H. Hamilton

Barton H. Hamilton

Washington University, Saint Louis - John M. Olin School of Business

Tat Y. Chan

Washington University in St. Louis - John M. Olin Business School

Multiple version iconThere are 2 versions of this paper

Date Written: July 2003

Abstract

Randomized experiments (REs) are viewed as the "gold standard" for the treatment evaluation, but many REs are plagued by attrition or non-compliance, even among subjects receiving the more effective treatment. This paper constructs an economic model of decision-making in which individuals make utility maximizing choices that provides a rich framework for evaluating REs. We estimate the subject's utility associated with the receipt of alternative treatments, as revealed by dropout or compliance behavior, to evaluate treatment effectiveness. Utility is a function of both the "publicly observed" outcomes that are typically the focus of evaluation studies, and treatment side effects that are the private information of the subject. Participants enter the RE uncertain of treatment effectiveness and often the treatment received, and update their prior beliefs over the course of the experiment when deciding whether to drop out. We use the framework to analyze an influential AIDS clinical trial, ACTG 175, which has been used to tout the benefits of combination therapies for AIDS over the use of AZT alone. However, our analysis indicates that for many subjects, AZT yields the highest level of utility, despite having the smallest impact on the publicly observed outcome of the study, the patient's CD4 count. Significant and rapid learning is observed over the course of the experiment, so that early dropout is primarily driven by side effects, while later attrition reflects declining CD4 counts for many subjects. An important implication of our findings, not recognized using the standard evaluation approach, is that patient welfare may be enhanced by offering a menu of therapies, since no single treatment is preferred by a majority of patients.

Keywords: Learning, Side Effects, Attrition, Treatment Evaluation, Randomized Experiments, Clinical Trial, Discrete Choice Dynamic Programming, Simulation Estimation, AIDS, ACTG 175

JEL Classification: I10, D83, C90

Suggested Citation

Hamilton, Barton H. and Chan, Tat Y., Learning, Private Information, and the Economic Evaluation of Randomized Experiments (July 2003). Olin School of Business Working Paper No. 2003-08-03. Available at SSRN: https://ssrn.com/abstract=436950 or http://dx.doi.org/10.2139/ssrn.436950

Barton H. Hamilton (Contact Author)

Washington University, Saint Louis - John M. Olin School of Business ( email )

One Brookings Drive, Campus Box 1133
St. Louis, MO 63130-4899
United States
314-935-8057 (Phone)
314-935-6359 (Fax)

HOME PAGE: http://www.olin.wustl.edu/faculty/hamiltonb/

Tat Y. Chan

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
94
Abstract Views
991
rank
294,866
PlumX Metrics