On De-Bunking 'Fake News' in the Post-Truth Era: How to Reduce Statistical Error in Research

Abstract

The authors note with alarm that statistical noise caused by statistical incompetence is beginning to creep into research on cost overrun in public investment projects, contaminating research with work that does not meet basic standards of validity and reliability. The paper gives examples of such work and proposes three heuristics to root out the problem. First, researchers who are not statisticians, or do not have a strong background in statistics, should abstain from doing statistical analysis, and instead rely on more experienced colleagues, preferably professional statisticians. Second, journal referees should clearly state their level of statistical proficiency to journal editors, so these can set the right referee team. Finally, journal editors should make sure that at least one referee is capable of reviewing the statistical and methodological aspects of a paper. The work under review would have benefitted from observing these simple heuristics, as would any work based on statistical analysis.

Keywords: public investment, transportation infrastructure, project management, behavioral science, statistical error, signal and noise, optimism bias, planning fallacy, Hiding Hand