Star Wars: The Empirics Strike Back

Paris School of Economics Working Paper No. 2012-29

38 Pages Posted: 23 Jun 2012

See all articles by Abel Brodeur

Abel Brodeur

IZA Institute of Labor Economics; University of Ottawa - Department of Economics

Mathias LÉ

Banque de France

Marc Sangnier

Aix-Marseille University - Aix-Marseille School of Economics

Yanos Zylberberg

University of Bristol

Multiple version iconThere are 2 versions of this paper

Date Written: June 18, 2012

Abstract

Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above .25, a valley between .25 and .10 and a bump slightly under .05. Missing tests are those which would have been accepted but close to being rejected (p-values between .25 and .10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10% and 20% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models.

Keywords: Hypothesis testing, distorting incentives, selection bias, research in economics

JEL Classification: A11, B41, C13, C44

Suggested Citation

Brodeur, Abel and Brodeur, Abel and LÉ, Mathias and Sangnier, Marc and Zylberberg, Yanos, Star Wars: The Empirics Strike Back (June 18, 2012). Paris School of Economics Working Paper No. 2012-29, Available at SSRN: https://ssrn.com/abstract=2089580 or http://dx.doi.org/10.2139/ssrn.2089580

Abel Brodeur (Contact Author)

University of Ottawa - Department of Economics ( email )

200 Wilbrod Street
Ottawa, Ontario K1N 6N5
Canada

HOME PAGE: http://https://sites.google.com/site/abelbrodeur/

IZA Institute of Labor Economics ( email )

P.O. Box 7240
Bonn, D-53072
Germany

Mathias LÉ

Banque de France ( email )

Paris
France

HOME PAGE: http://https://sites.google.com/site/mathiasle/home

Marc Sangnier

Aix-Marseille University - Aix-Marseille School of Economics ( email )

5-9 Boulevard Bourdet
Marseille, 13001
France

Yanos Zylberberg

University of Bristol ( email )

Avon

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,035
Abstract Views
10,328
Rank
32,548
PlumX Metrics