The State of Social and Personality Science: Rotten to the Core, Not So Bad, Getting Better, or Getting Worse?

Abstract

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the prevalence of questionable research practices (QRPs) and the replicability of particular findings, the impact of the current discussion on research practices is unknown. The current studies examine whether and how practices have changed, if at all, over the last 10 years. In Study 1, we surveyed 1,166 social and personality psychologists about how the current debate has affected their perceptions of their own and the field’s research practices. In Study 2, we coded the research practices and critical test statistics from social and personality psychology articles published in 2003-2004 and 2013-2014.

Together, these studies suggest that:

(1) perceptions of the current state of the field are more pessimistic than optimistic;

(2) the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices,

(3) the estimated replicability of research published in 2003-2004 may not be as bad as many feared, and

(4) research published in 2013-2014 shows some improvement over research published in 2003-2004, a result that suggests the field is evolving in a positive direction.

Keywords: Meta-Science, Questionable Research Practices, Replicability