Star Wars: The Empirics Strike Back

61 Pages Posted: 23 Mar 2013

See all articles by Abel Brodeur

Abel Brodeur

University of Ottawa - Department of Economics; IZA Institute of Labor Economics

Mathias LÉ

Autorité de Contrôle Prudentiel et de Résolution (ACPR)

Marc Sangnier

Aix-Marseille University - Aix-Marseille School of Economics

Yanos Zylberberg

University of Bristol

Multiple version iconThere are 2 versions of this paper

Abstract

Journals favor rejection of the null hypothesis. This selection upon tests may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above 0.25, a valley between 0.25 and 0.10 and a bump slightly below 0.05. The missing tests (with p-values between 0.25 and 0.10) can be retrieved just after the 0.05 threshold and represent 10% to 20% of marginally rejected tests. Our interpretation is that researchers might be tempted to inflate the value of those almost-rejected tests by choosing a "significant" specification. We propose a method to measure inflation and decompose it along articles' and authors' characteristics.

Keywords: hypothesis testing, distorting incentives, selection bias, research in economics

JEL Classification: A11, B41, C13, C44

Suggested Citation

Brodeur, Abel and LÉ, Mathias and Sangnier, Marc and Zylberberg, Yanos, Star Wars: The Empirics Strike Back. IZA Discussion Paper No. 7268. Available at SSRN: https://ssrn.com/abstract=2238281

Abel Brodeur (Contact Author)

University of Ottawa - Department of Economics ( email )

200 Wilbrod Street
Ottawa, Ontario K1N 6N5
Canada

HOME PAGE: http://https://sites.google.com/site/abelbrodeur/

IZA Institute of Labor Economics ( email )

P.O. Box 7240
Bonn, D-53072
Germany

Mathias LÉ

Autorité de Contrôle Prudentiel et de Résolution (ACPR) ( email )

61, rue Taitbout
Paris, 75436
France

Marc Sangnier

Aix-Marseille University - Aix-Marseille School of Economics ( email )

5-9 Boulevard Bourdet
Marseille, 13001
France

Yanos Zylberberg

University of Bristol ( email )

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
93
Abstract Views
1,201
rank
23,619
PlumX Metrics