The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality

Abstract

This paper is concerned with the influence of scientists’ prior beliefs on their judgments of evidence quality. A laboratory experiment using advanced graduate students in the sciences (study 1) and an experimental survey of practicing scientists on opposite sides of a controversial issue (study 2) revealed agreement effects. Research reports that agreed with scientists’ prior beliefs were judged to be of higher quality than those that disagreed. In study 1, a prior belief strength X agreement interaction was found, indicating that the agreement effect was larger for general, evaluative judgments than for more specific, analytical judgments. A Bayesian analysis indicates that the pattern of agreement effects found in these studies may be normatively defensible, although arguments against implementing a Bayesian approach to scientific judgment are also advanced.

Keywords: Prior belief, judgment, Bayesian