Visualizing Uncertainty: On Soyer's and Hogarth's 'The Illusion of Predictability: How Regression Statistics Mislead Experts'

Abstract

This comment was published in the International Journal of Forecasting symposium on the Soyer-Hogarth experiment (Vol. 28, No. 3, July/Sept. 2012, pp. 712-714). The experiment evaluates the ability of expert econometricians to make predictions based on commonly provided regression output. Visual displays of quantitative information, including simple plots of data, outperformed predictions based on R-squared, t-statistics, and other common diagnostics. Reliance on graphing - on the visualization of uncertainty - was suggested more than a century ago by Karl Pearson, a founding father of English language statistics. The results of the Soyer and Hogarth experiment, when combined with evidence produced by Ziliak and McCloskey (2008) and others, suggests that graphing and visualization should receive more attention and tests of statistical significance, less.

Keywords: regression analysis, forecasting, uncertainty, visualization, statistical significance, Karl Pearson

JEL Classification: C1, C18, B1