Performance on the Cognitive Reflection Test is Stable Across Time

8 Pages Posted: 13 Feb 2018 Last revised: 9 Jun 2018

Michael Stagnaro

Yale University - Department of Psychology

Gordon Pennycook

University of Regina

David G. Rand


Date Written: January 31, 2018


A widely used measure of individual propensity to utilize analytic processing is the Cognitive Reflection Test (CRT), a set of math problems with intuitively compelling but incorrect answers. Here, we ask whether scores on this measure are temporally stable. We aggregate data from 11 studies run on Amazon Mechanical Turk in which the Cognitive Reflection Test (CRT) was administered and identify N = 3,302 unique individuals who completed the CRT two or more times. We find a strong correlation between an individual’s first and last CRT performance, r = .806. This remains true even when constraining to data points separated by over 2 years, r = .755. Furthermore, we find that CRT scores from one timepoint correlated negatively with belief in God and social conservatism from the other timepoint (and to a similar extent as scores gathered at the same timepoint). These results show that CRT scores are stable over time, and – given the stable relationship between CRT and religious belief and ideology – provide some evidence for the stability of analytic cognitive style more generally.

Keywords: cognitive reflection test, analytic cognitive style, dual process, intuition, reflection, religious belief, ideology

Suggested Citation

Stagnaro, Michael and Pennycook, Gordon and Rand, David G., Performance on the Cognitive Reflection Test is Stable Across Time (January 31, 2018). Available at SSRN: or

Michael Stagnaro (Contact Author)

Yale University - Department of Psychology ( email )

P.O. Box 208205
New Haven, CT 06520-8205
United States

Gordon Pennycook

University of Regina ( email )

3737 Wascana Parkway
Regina, Saskatchewan S4S OA2

David G. Rand

MIT ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States


Register to save articles to
your library


Paper statistics

Abstract Views