Practical Significance of Statistical Significance
22 Pages Posted: 17 Aug 2017
Date Written: August 16, 2017
Abstract
Statistical significance is a core concept in many sciences to distinguish between what is and what is not. But what are the practical implications of finding a significant effect, and is it more reliable when based on more observations or higher confidence levels? We consider forecast accuracy in a practical setting using significant monthly deviations from the mean in a more than 300 year long time series of stock market returns. While statistical significance in general implies a marginal improved probability of correct forecasts, results vary strongly depending on significance levels and number of observations used. Surprisingly, signals based on higher significance levels or more observations do not necessarily imply better forecasts.
Keywords: Return predictability, January effect, forecasting
JEL Classification: G10
Suggested Citation: Suggested Citation