Communicating the Robustness of Findings of COVID-19 Studies

11 Pages Posted: 12 Jun 2020

See all articles by Ken Frank

Ken Frank

Michigan State University

Qinyun Lin

University of Chicago

Spiro Maroulis

Arizona State University (ASU) - School of Public Affairs

Ran Xu

University of Connecticut

Joshua M. Rosenberg

University of Tennessee, Knoxville

Christopher S. Hayter

Center for Organization Research and Design

Ramy A. Mahmoud

Optinose, Inc.

Maryina Kolak

University of Chicago

Thomas Dietz

Michigan State University, Department of Sociology and Environmental Science and Policy Program

Lixin Zhang

Michigan State University

Date Written: May 22, 2020

Abstract

The COVID-19 pandemic is forcing researchers, clinicians, and policymakers to accelerate the evaluation of treatments and vaccines. Critical to these evaluations is the ability to characterize the uncertainty of inferences in clear terms accessible to a broad set of stakeholders with varying statistical backgrounds. Here we quantify the uncertainty about inferences from Randomized Controlled Trials (RCTs) by quantifying how many patients would have to have experienced different outcomes to change the inference. For example, the inference of a positive effect of Hydroxychloroquine (HCQ) on pneumonia from an open label RCT would be overturned if one of the treatment cases characterized as improved had instead been characterized as unchanged or exacerbated. We generalize the technique to apply to thresholds defined by any effect size. We also apply the analysis to an inference of no effect of Remdesivir on mortality and an historical example of anti-hypertensive treatments on stroke. Quantifying the robustness of inferences in terms of patient outcomes supports a more precise dialogue among clinicians, researchers, policymakers, and the general public.

Note: Funding: None to declare

Declaration of Interest: None to declare

Keywords: COVID-19 treatments, causal inference, robustness, policy

JEL Classification: C12, C19

Suggested Citation

Frank, Ken and Lin, Qinyun and Maroulis, Spiro and Xu, Ran and Rosenberg, Joshua M. and Hayter, Christopher S. and Mahmoud, Ramy A. and Kolak, Maryina and Dietz, Thomas and Zhang, Lixin, Communicating the Robustness of Findings of COVID-19 Studies (May 22, 2020). Available at SSRN: https://ssrn.com/abstract=3607967

Ken Frank (Contact Author)

Michigan State University

Qinyun Lin

University of Chicago

1101 East 58th Street
Chicago, IL 60637
United States

Spiro Maroulis

Arizona State University (ASU) - School of Public Affairs ( email )

411 N. Central Ave
Phoenix, AZ 85004
United States

HOME PAGE: http://www.public.asu.edu/~smarouli/

Ran Xu

University of Connecticut ( email )

Joshua M. Rosenberg

University of Tennessee, Knoxville

The Boyd Center for Business and Economic Research
Knoxville, TN 37996
United States

Christopher S. Hayter

Center for Organization Research and Design ( email )

School of Public Affiars
441 N. Central Ave.
Phoenix, AZ 85004
United States

Ramy A. Mahmoud

Optinose, Inc.

Maryina Kolak

University of Chicago

1101 East 58th Street
Chicago, IL 60637
United States

Thomas Dietz

Michigan State University, Department of Sociology and Environmental Science and Policy Program ( email )

East Lansing, MI 48824
United States
517-353-8763 (Phone)

Lixin Zhang

Michigan State University

Agriculture Hall
East Lansing, MI 48824-1122
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
57
Abstract Views
373
rank
403,811
PlumX Metrics