Sampling and the curse of the case study‎

Abstract

Regardless of how the outcomes of a given P/CVE program are measured or evaluated, a ‎fundamental, implicit (if not explicit) research question is: to what extent can obtained ‎results apply to others within a given population. In short, to what extent can the results apply ‎to others, in general; what is the so-called generalizability of the findings? In other words, ‎the outcomes of a given P/CVE program are relatively useless unless they can be replicated, and ‎the likelihood of replication is synonymous with generalizability. Therefore, it is virtually ‎impossible to overstate the importance of generalizability with respect to P/CVE research ‎and evaluation, and generalizability is fundamentally a function of how well sampling is ‎performed. Therefore, it is also virtually impossible to overstate the importance of sampling with ‎respect to an evidence-based approach to P/CVE. This research methods brief describes ‎fundamental issues (including potential pitfalls and means to avoid them) with respect to ‎sampling in the context of P/CVE program design and evaluation: including issues related to ‎sampling online “Big Data,” and “nested” (multi-level/hierarchical) program/research designs.‎

Keywords: Preventing Violent Extremism, PVE, Countering Violent Extremism, CVE, P/CVE, Design, Evaluation, Sampling