A Larger-N, Fewer Variables Problem? The Counterintuitive Sensitivity of QCA
Qualitative & Multi-Method Research, Spring 2014, Vol. 12, No. 1
9 Pages Posted: 23 Jan 2015
Date Written: January 20, 2015
Studies employing Qualitative Comparative Analysis (QCA) often analyze a relatively small number of cases (n) to assess the impact of a substantial number of variables (k) on a given outcome. In the tradition of writing on the comparative method and multi-method research, however, this ratio of cases-to-variables is viewed as an analytic problem. In this exploratory research note, we raise questions about the implications of this ratio for the stability of findings in QCA. One common method of assessing result stability is the “drop-one” sensitivity test, which repeatedly reruns a particular analysis, each time dropping a single case. We find that, for the number of cases to which analysts most routinely apply QCA, this type of sensitivity analysis produces paradoxical results. Directly contrary to the standard expectation that more robust findings emerge with a higher n/k ratio, we discover that QCA findings based on a lower n/k ratio prove to be more stable. This result calls into question the validity of the drop-one test as a sensitivity metric for QCA.
Keywords: Qualitative Comparative Analysis, qualitative methodology, sensitivity tests, simulation methods
JEL Classification: C10, C15, C63
Suggested Citation: Suggested Citation