P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results

16 Pages Posted: 11 Jan 2014 Last revised: 19 Nov 2014

See all articles by Uri Simonsohn

Uri Simonsohn

Ramon Llull University - ESADE Business School

Leif D. Nelson

University of California, Berkeley - Haas School of Business

Joseph P. Simmons

University of Pennsylvania - The Wharton School

Date Written: April 27, 2014

Abstract

Journals tend to publish only statistically significant evidence, creating a scientific record that markedly overstates the size of effects. We provide a new tool that corrects for this bias without requiring access to nonsignificant results. It capitalizes on the fact that the distribution of significant p-values, p-curve, is a function of the true underlying effect. Researchers armed only with sample sizes and test results of the published findings can correct for publication bias. We validate the technique with simulations and by re-analyzing data from the Many-Labs Replication project. We demonstrate that p-curve can arrive at conclusions opposite that of existing tools by re-analyzing the meta-analysis of the “choice overload” literature.

Keywords: Publication bias, p-curve

Suggested Citation

Simonsohn, Uri and Nelson, Leif D. and Simmons, Joseph P., P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results (April 27, 2014). Available at SSRN: https://ssrn.com/abstract=2377290 or http://dx.doi.org/10.2139/ssrn.2377290

Uri Simonsohn (Contact Author)

Ramon Llull University - ESADE Business School ( email )

Avinguda de la Torre Blanca, 59
Sant Cugat del Vallès, 08172
Spain

HOME PAGE: http://urisohn.com

Leif D. Nelson

University of California, Berkeley - Haas School of Business ( email )

545 Student Services Building, #1900
2220 Piedmont Avenue
Berkeley, CA 94720
United States

Joseph P. Simmons

University of Pennsylvania - The Wharton School ( email )

3733 Spruce Street
Philadelphia, PA 19104-6374
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
6,166
Abstract Views
26,126
Rank
1,994
PlumX Metrics