Download this Paper Open PDF in Browser

The Crisis of Confidence in Research Findings in Psychology: Is Lack of Replication the Real Problem? Or is it Something Else?

Archives of Scientific Psychology (Forthcoming)

Fox School of Business Research Paper No. 16-036

22 Pages Posted: 3 May 2016 Last revised: 16 Dec 2016

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business

In‐Sue Oh

Temple University - Department of Human Resource Management

Date Written: May 2, 2016

Abstract

There have been frequent expressions of concern over the supposed failure of researchers to conduct replication studies. But the large number of meta-analyses in our literatures shows that replication studies are in fact being conducted in most areas of research. Many who argue for replication as the “gold standard” consider a non-significant replication attempt to be strong evidence against the initial study, an interpretation that ignores statistical power, typically low in behavioral research. Many researchers also hold that there is no need to replicate a non-significant finding, believing it will always replicate perfectly, an erroneous belief. These beliefs lead to a widely accepted sequential model of the research process that is deficient because it assumes that a single study can answer a research question, a belief that meta-analysis has shown to be false. Meta-analysis can provide the solution to these problems if the problems of publication bias and questionable research practices are successfully addressed. The real problem is not a lack of replication; it is the distortion of our research literatures caused by publication bias and questionable research practices.

Keywords: Replication; Meta-Analysis; Publication Bias; Questionable Research Practices

Suggested Citation

Schmidt, Frank L. and Oh, In‐Sue, The Crisis of Confidence in Research Findings in Psychology: Is Lack of Replication the Real Problem? Or is it Something Else? (May 2, 2016). Archives of Scientific Psychology (Forthcoming); Fox School of Business Research Paper No. 16-036. Available at SSRN: https://ssrn.com/abstract=2773582

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business ( email )

Acquisitions
5020 Main Library
Iowa City, IA 52242-1000
United States

In-Sue Oh (Contact Author)

Temple University - Department of Human Resource Management ( email )

1801 Liacouras Walk
Alter Hall 343
Philadelphia, PA 19122
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/in-sue-oh/

Paper statistics

Downloads
155
Rank
161,223
Abstract Views
638