Correcting Between-Participant Discourse Bias in Comment Classification
Posted: 5 May 2015 Last revised: 16 Apr 2017
Date Written: May 3, 2015
Text mining and natural language processing have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment classification, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment classification is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well known tf-idf formula to compute these weights.
This paper reveals the bias introduced by between-participants' discourse to the study of comments in social media, and proposes a correction. We find that content extracted from between-participants' discourse is often highly correlated, resulting in dependency structures between observations in the study. Ignoring this bias can manifest in a non-robust analysis at best, and can lead to an entirely wrong conclusion at worst. We propose a statistical correction to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with real data from Facebook.
Keywords: tf-idf, discourse, bias, comments classification, sentiment analysis, opinion mining, subjectivity, information leakage
Suggested Citation: Suggested Citation