Lady Justice v. Cult of Statistical Significance: Oomph-Less Science and the New Rule of Law
Forthcoming, Oxford Handbook on Professional Economic Ethics (Oxford University Press, 2014), edited by G. DeMartino and D.N. McCloskey
25 Pages Posted: 22 Jul 2014
Date Written: July 17, 2014
We have an ethical problem in economics and other sciences using null hypothesis statistical significance testing without a loss function — a test that avoids asking, How Big is a Big Loss or Gain? Statistical significance is not equivalent to economic significance, nor to medical, clinical, nor any other kind of scientific significance — those functions of gain and loss. The mistake in the falsely made equation is evident when one reflects that the estimated payoff from a lottery is not the same object as the odds of winning that lottery. Yet a widespread failure to make the distinction between an estimate of human consequence and an estimate of its probability — between the meaning of an estimated average and the random variance around it — is killing people in medicine and impoverishing people in economics (Ziliak and McCloskey 2008). The ethical problem created by a test of statistical significance is made worse by the method’s blatant illogic at the foundational level, a fact that is unacknowledged by the bulk of decision makers depending upon it. Several changes to the scientific paper — and a recent decision by the Supreme Court of the United States — could help.
Keywords: ethics, statistical significance, oomph, Matrixx v. Siracusano, Fisher, Gosset
JEL Classification: C10, C12, C13, B23, A13
Suggested Citation: Suggested Citation