Choosing the Level of Significance: A Decision-Theoretic Approach

53 Pages Posted: 30 Aug 2015 Last revised: 6 Apr 2019

See all articles by Jae H. Kim

Jae H. Kim

La Trobe University - School of Economics and Finance

In Choi

Sogang University

Date Written: April 5, 2019

Abstract

In many areas of science including business disciplines, statistical decisions are often made almost exclusively at a conventional level of significance. Serious concerns have been raised that this contributes to a range of ill-practices such as p-hacking and data-mining that undermine research credibility. In this paper, we present a decision-theoretic approach to choosing the optimal level of significance, with a consideration of the key factors of hypothesis testing, including sample size, prior belief, and losses from Type I and II errors. We present the method in the context of testing for linear restrictions in the linear regression model. From the empirical applications in accounting, economics, and finance, we find that the decisions made at the optimal significance levels are more sensible and unambiguous than those at a conventional level, providing inferential outcomes consistent with estimation results, descriptive analysis, and economic reasoning. Computational resources are provided with two R packages.

Keywords: Bootstrapping, Expected Loss, Statistical Significance, Power Analysis

JEL Classification: A20, C12, B41

Suggested Citation

Kim, Jae H. and Choi, In, Choosing the Level of Significance: A Decision-Theoretic Approach (April 5, 2019). Available at SSRN: https://ssrn.com/abstract=2652773 or http://dx.doi.org/10.2139/ssrn.2652773

Jae H. Kim (Contact Author)

La Trobe University - School of Economics and Finance ( email )

Department of Finance
La Trobe Business School
Bundoora, IN 3086
Australia

In Choi

Sogang University ( email )

Seoul 121-742
Korea, Republic of (South Korea)

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
286
Abstract Views
1,306
rank
111,737
PlumX Metrics