Rejection Odds and Rejection Ratios: A Proposal for Statistical Practice in Testing Hypotheses

37 Pages Posted: 13 Jan 2016

See all articles by Maestro Bayarri Jorge

Maestro Bayarri Jorge

University of Valencia

Daniel J. Benjamin

Anderson School of Management; USC, Center for Economic and Social Research (CESR); National Bureau of Economic Research (NBER); Human Genetics Department, David Geffen School of Medicine

James Berger

Duke University

Thomas Sellke

Purdue University

Date Written: December 1, 2015

Abstract

Much of science is (rightly or wrongly) driven by hypothesis testing. Even in situations where the hypothesis testing paradigm is correct, the common practice of basing inferences solely on p-values has been under intense criticism for over 50 years. We propose, as an alternative, the use of the odds of a correct rejection of the null hypothesis to incorrect rejection. Both pre-experimental versions (involving the power and Type I error) and post-experimental versions (depending on the actual data) are considered. Implementations are provided that range from depending only on the p-value to consideration of full Bayesian analysis. A surprise is that all implementations — even the full Bayesian analysis — have complete frequentist justification. Versions of our proposal can be implemented that require only minor modifications to existing practices yet overcome some of their most severe shortcomings.

Keywords: hypothesis testing, p-value, power, Bayes factor

JEL Classification: C11, C12, C90

Suggested Citation

Jorge, Maestro Bayarri and Benjamin, Daniel J. and Berger, James and Sellke, Thomas, Rejection Odds and Rejection Ratios: A Proposal for Statistical Practice in Testing Hypotheses (December 1, 2015). CESR-Schaeffer Working Paper No. 2016-002, Available at SSRN: https://ssrn.com/abstract=2714185 or http://dx.doi.org/10.2139/ssrn.2714185

Maestro Bayarri Jorge

University of Valencia ( email )

Avda. de los Naranjos s/n
Valencia, Valencia E-46022
Spain

Daniel J. Benjamin (Contact Author)

Anderson School of Management ( email )

110 Westwood Plaza
Los Angeles, CA 90095-1481
United States

USC, Center for Economic and Social Research (CESR) ( email )

635 Downey Way
Los Angeles, CA 90089-3332
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Human Genetics Department, David Geffen School of Medicine ( email )

695 Charles E. Young Drive South
Los Angeles, CA 90095-7088
United States

James Berger

Duke University ( email )

100 Fuqua Drive
Durham, NC 27708-0204
United States

Thomas Sellke

Purdue University ( email )

610 Purdue Mall
West Lafayette, IN 47907
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
53
Abstract Views
554
rank
436,836
PlumX Metrics