Statistical Decision Properties of Imprecise Trials Assessing Covid-19 Drugs

30 Pages Posted: 10 Jun 2020 Last revised: 11 Jun 2020

See all articles by Charles F. Manski

Charles F. Manski

Northwestern University - Department of Economics; National Bureau of Economic Research (NBER)

Aleksey Tetenov

University of Geneva

Date Written: June 2020

Abstract

As the COVID-19 pandemic progresses, researchers are reporting findings of randomized trials comparing standard care with care augmented by experimental drugs. The trials have small sample sizes, so estimates of treatment effects are imprecise. Seeing imprecision, clinicians reading research articles may find it difficult to decide when to treat patients with experimental drugs. Whatever decision criterion one uses, there is always some probability that random variation in trial outcomes will lead to prescribing sub-optimal treatments. A conventional practice when comparing standard care and an innovation is to choose the innovation only if the estimated treatment effect is positive and statistically significant. This practice defers to standard care as the status quo. To evaluate decision criteria, we use the concept of near optimality, which jointly considers the probability and magnitude of decision errors. An appealing decision criterion from this perspective is the empirical success rule, which chooses the treatment with the highest observed average patient outcome in the trial. Considering the design of recent and ongoing COVID-19 trials, we show that the empirical success rule yields treatment results that are much closer to optimal than those generated by prevailing decision criteria based on hypothesis tests.

Suggested Citation

Manski, Charles F. and Tetenov, Aleksey, Statistical Decision Properties of Imprecise Trials Assessing Covid-19 Drugs (June 2020). NBER Working Paper No. w27293, Available at SSRN: https://ssrn.com/abstract=3621797

Charles F. Manski (Contact Author)

Northwestern University - Department of Economics ( email )

2003 Sheridan Road
Evanston, IL 60208
United States
847-491-8223 (Phone)
847-491-7001 (Fax)

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Aleksey Tetenov

University of Geneva

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
8
Abstract Views
33
PlumX Metrics