Table of Contents

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA

Chris Krogslund, University of California, Berkeley
Donghyun Danny Choi, University of California, Berkeley
Mathias Poertner, University of California, Berkeley - Charles and Louise Travers Department of Political Science

Signals of Public Opinion in Online Communication: A Comparison of Methods and Data Sources

Sandra Gonzalez-Bailon, University of Pennsylvania - Annenberg School for Communication
Georgios Paltoglou, University of Wolverhampton


POLITICAL METHODS: COMPUTATIONAL eJOURNAL

"Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA" Free Download
Political Analysis (2015) 23:21-41

CHRIS KROGSLUND, University of California, Berkeley
Email:
DONGHYUN DANNY CHOI, University of California, Berkeley
Email:
MATHIAS POERTNER, University of California, Berkeley - Charles and Louise Travers Department of Political Science
Email:

Scholars have increasingly turned to fuzzy set Qualitative Comparative Analysis (fsQCA) to conduct small and medium-N studies, arguing that it combines the most desired elements of variable-oriented and case oriented research. This article demonstrates, however, that fsQCA is an extraordinarily sensitive method whose results are worryingly susceptible to minor parametric and model specification changes. We make two specific claims. First, the causal conditions identified by fsQCA as being sufficient for an outcome to occur are highly contingent upon the values of several key parameters selected by the user. Second, fsQCA results are subject to marked confirmation bias. Given its tendency toward finding complex connections between variables, the method is highly likely to identify as sufficient for an outcome causal combinations containing even randomly generated variables. To support these arguments, we replicate three articles utilizing fsQCA and conduct sensitivity analyses and Monte Carlo simulations to assess the impact of small changes in parameter values and the method’s built-in confirmation bias on the overall conclusions about sufficient conditions.

"Signals of Public Opinion in Online Communication: A Comparison of Methods and Data Sources" Free Download
The Annals of the American Academy of Political and Social Science, Forthcoming

SANDRA GONZALEZ-BAILON, University of Pennsylvania - Annenberg School for Communication
Email:
GEORGIOS PALTOGLOU, University of Wolverhampton
Email:

This study offers a systematic comparison of automated content analysis tools by assessing their ability to correctly identify affective tone (e.g., positive vs. negative) in different data contexts and social media environments. Our comparisons assess the reliability and validity of publicly available, off-the-shelf classifiers. We use datasets from a range of online sources that vary in the diversity and formality of the language used, and we apply different classifiers to extract information about the affective tone in these datasets. We first measure agreement (reliability test) and then compare their classifications with the benchmark of human coding (validity test). Our analyses show that validity and reliability vary with the formality and diversity of the text; we also show that ready-to-use methods leave much space for improvement in domain-specific content and that a machine-learning approach offers more accurate predictions.

^top

About this eJournal

This eJournal distributes working and accepted paper abstracts. Papers in this area study the formation, structure and function of networks or apply computational methods or algorithmic game theory to understanding politics.

Submissions

To submit your research to SSRN, sign in to the SSRN User HeadQuarters, click the My Papers link on left menu and then the Start New Submission button at top of page.

Distribution Services

If your organization is interested in increasing readership for its research by starting a Research Paper Series, or sponsoring a Subject Matter eJournal, please email: RPS@SSRN.com

Distributed by

Political Science Network (PSN), a division of Social Science Electronic Publishing (SSEP) and Social Science Research Network (SSRN)

Directors

POLITICAL METHODS EJOURNALS

DAVID A. LAKE
UC San Diego
Email: dlake@ucsd.edu

MATHEW D. MCCUBBINS
Duke University
Email: mathew.mccubbins@duke.edu

Please contact us at the above addresses with your comments, questions or suggestions for PSN-Sub.