Small Telescopes: Detectability and the Evaluation of Replication Results

27 Pages Posted: 3 May 2013 Last revised: 17 Dec 2014

See all articles by Uri Simonsohn

Uri Simonsohn

ESADE Business School; Ramon Llull University - ESADE Business School; ESADE Business School

Date Written: December 16, 2014

Abstract

This paper introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, assessing the extent to which the replication results are consistent with an effect size big enough to be detectable by the original study. The approach is demonstrated revisiting replications of three famous findings: the embodiment of morality, the endowment effect, and weather & happiness. Its benefits include: (i) distinguishing p>.05 replications that are simply too-noisy from those that actively indicate the effect is undetectably different from zero, (ii) “protecting” true findings from underpowered replications, and (iii) arriving at intuitively compelling inferences in general and for the revisited replications in particular.

Keywords: Replications, power, p-values, effect size

Suggested Citation

Simonsohn, Uri, Small Telescopes: Detectability and the Evaluation of Replication Results (December 16, 2014). Psychological Science, Forthcoming, Available at SSRN: https://ssrn.com/abstract=2259879 or http://dx.doi.org/10.2139/ssrn.2259879

Uri Simonsohn (Contact Author)

ESADE Business School ( email )

Av. de Pedralbes, 60-62
Barcelona, 08034
Spain

Ramon Llull University - ESADE Business School ( email )

Avinguda de la Torre Blanca, 59
Sant Cugat del Vallès, 08172
Spain

HOME PAGE: http://urisohn.com

ESADE Business School ( email )

Av. de Pedralbes, 60-62
Barcelona, 08034
Spain

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
2,784
Abstract Views
21,656
Rank
8,713
PlumX Metrics