The IC Rating Scale as a Measure of Analytic Rigor

41 Pages Posted: 27 Jul 2023 Last revised: 28 Jul 2023

See all articles by Luke Thorburn

Luke Thorburn

King's College London

Timothy van Gelder

University of Melbourne

Christine Shahan Brugh

North Carolina State University

Sue Mi Kim

North Carolina State University

Michele Kolb

North Carolina State University

Barry Zulauf

Georgetown University

Date Written: May 14, 2022

Abstract

Analytic rigor is a cardinal virtue of intelligence analysis. However, there is no widely used, well-developed measure of rigor. An alternative is to use an existing quality measure, the ODNI’s IC Rating Scale. We investigated the extent to which the Scale is a measure of analytic rigor by estimating the correlation between Scale scores and rigor scores on 100 fictional but realistic intelligence products. The correlation was 0.42, or weak-to-moderate. This suggests that Scale scores and rigor can diverge, and that the intelligence community should not rely solely on the Scale for assessing rigor.

Keywords: analytic rigor, IC Rating Scale, validity, reliability

Suggested Citation

Thorburn, Luke and van Gelder, Timothy and Shahan Brugh, Christine and Kim, Sue Mi and Kolb, Michele and Zulauf, Barry, The IC Rating Scale as a Measure of Analytic Rigor (May 14, 2022). Available at SSRN: https://ssrn.com/abstract=4519926 or http://dx.doi.org/10.2139/ssrn.4519926

Luke Thorburn (Contact Author)

King's College London ( email )

United Kingdom

Timothy Van Gelder

University of Melbourne ( email )

185 Pelham Street
Carlton, Victoria 3053
Australia

Christine Shahan Brugh

North Carolina State University ( email )

Hillsborough Street
Raleigh, NC 27695
United States

Sue Mi Kim

North Carolina State University

Hillsborough Street
Raleigh, NC 27695
United States

Michele Kolb

North Carolina State University

Hillsborough Street
Raleigh, NC 27695
United States

Barry Zulauf

Georgetown University

Washington, DC 20057
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
71
Abstract Views
316
Rank
614,265
PlumX Metrics