Reforming Subgroup Analysis

17 Pages Posted: 14 Apr 2008

See all articles by Anup Malani

Anup Malani

University of Chicago - Law School; University of Chicago Pritzker School of Medicine; Resources for the Future; National Bureau of Economic Research (NBER)

Oliver Bembom

University of California, Berkeley

Mark van der Laan

University of California, Berkeley

Date Written: April 13, 2008

Abstract

The FDA largely approves or disapproves drugs based on average treatment effects. Given widespread heterogeneity in treatment response, this approach can result in the approval of drugs with significant negative effects for identifiable subgroups (false positives) and in the non-approval of drugs with significant positive effects for identifiable subgroups (false negatives). Despite the FDA's position, drug companies frequently conduct post hoc subgroup analysis - a search for responsive subgroups - after their clinical trials find no positive average treatment effects. The FDA rejects such analysis due to the risk of spurious results. With sufficient covariate measurements, a drug company can always find some subgroup that benefits from a drug. This paper asks whether there workable compromise between the FDA and drug companies. Specifically, we seek a drug approval process that can use post hoc subgroup analysis to eliminate false negatives but does not risk opportunistic behavior and spurious correlation. The primary reform we recommend is a statistical analysis of a random subset of the data set from a clinical trial by an independent researcher. The subsample examined by the independent researcher can eliminates the risk of spurious findings due to multiple testing in the remainder of the sample. We apply our approach to the results of a recent clinical trial of a cancer drug, Xcytrin, that failed to find positive average treatment effects, and discover positive treatment effects for an important subset of patients.

Suggested Citation

Malani, Anup and Bembom, Oliver and van der Laan, Mark, Reforming Subgroup Analysis (April 13, 2008). Available at SSRN: https://ssrn.com/abstract=1119970 or http://dx.doi.org/10.2139/ssrn.1119970

Anup Malani (Contact Author)

University of Chicago - Law School ( email )

1111 E. 60th St.
Chicago, IL 60637
United States
773-702-9602 (Phone)
773-702-0730 (Fax)

HOME PAGE: http://www.law.uchicago.edu/faculty/malani/

University of Chicago Pritzker School of Medicine

Chicago, IL 60637
United States

Resources for the Future

1616 P Street, NW
Washington, DC 20036
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Oliver Bembom

University of California, Berkeley

310 Barrows Hall
Berkeley, CA 94720
United States

Mark Van der Laan

University of California, Berkeley

310 Barrows Hall
Berkeley, CA 94720
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
165
Abstract Views
1,397
rank
183,553
PlumX Metrics