Replication Unreliability in Psychology: Elusive Phenomena or 'Elusive' Statistical Power?

Tressoldi PE (2012) Replication Unreliability in Psychology: Elusive Phenomena or 'Elusive' Statistical Power? Front. Psychology 3:218. DOI: 10.3389/fpsyg.2012.00218

Posted: 5 Jul 2012

Date Written: July 4, 2012

Abstract

The focus of this paper is to analyze whether the unreliability of results related to certain controversial psychological phenomena may be a consequence of their low statistical power. Applying the Null Hypothesis Statistical Testing (NHST), still the widest used statistical approach, unreliability derives from the failure to refute the null hypothesis, in particular when exact or quasi-exact replications of experiments are carried out. Taking as example the results of meta-analyses related to four different controversial phenomena, subliminal semantic priming, incubation effect for problem solving, unconscious thought theory, and non-local perception, it was found that, except for semantic priming on categorization, the statistical power to detect the expected effect size (ES) of the typical study, is low or very low. The low power in most studies undermines the use of NHST to study phenomena with moderate or low ESs. We conclude by providing some suggestions on how to increase the statistical power or use different statistical approaches to help discriminate whether the results obtained may or may not be used to support or to refute the reality of a phenomenon with small ES.

Keywords: incubation effect, non-local perception, power, subliminal priming, unconscious thought theory

Suggested Citation

Tressoldi, Patrizio E., Replication Unreliability in Psychology: Elusive Phenomena or 'Elusive' Statistical Power? (July 4, 2012). Tressoldi PE (2012) Replication Unreliability in Psychology: Elusive Phenomena or 'Elusive' Statistical Power? Front. Psychology 3:218. DOI: 10.3389/fpsyg.2012.00218. Available at SSRN: https://ssrn.com/abstract=2100479

Patrizio E. Tressoldi (Contact Author)

Università di Padova ( email )

via Venezia 8
Padova, 35131
Italy

HOME PAGE: http://www.patriziotressoldi.it

Register to save articles to
your library

Register

Paper statistics

Abstract Views
87
PlumX Metrics