Practical Significance, Meta-Analysis and the Credibility of Economics

Abstract

Recently, there has been much discussion about replicability and credibility. By integrating the full research record, increasing statistical power, reducing bias and enhancing credibility, meta-analysis is widely regarded as 'best evidence'. Through Monte Carlo simulation, closely calibrated on the typical conditions found among 6,700 economics research papers, we find that large biases and high rates of false positives will often be found by conventional meta-analysis methods. Nonetheless, the routine application of meta-regression analysis and considerations of practical significance largely restore research credibility.

Keywords: meta-analysis, meta-regression, publication bias, credibility, simulations

JEL Classification: C10, C12, C13, C40