Detecting ‘Dirt’ and ‘Toxicity’: Rethinking Content Moderation as Pollution Behaviour

Abstract

"Dirt" and "toxicity" have become established ways of understanding and describing harmful content online. While the concepts have becoming naturalised as a way of expressing harm, its uses are also mobilised to very different ends. We wish to use this article to explore the role the notions of "toxic" and "dirt" have come to play in online content moderation, and what the politics of this discourse implies. Our overarching aim is to ask how content moderation infrastructures define toxic and dirty content and what the politics of these definitions are. We argue that content moderation’s historical reliance on binary categories – and those categories’ ongoing entanglements with social systems of racism and patriarchy – embeds the infrastructures in structures that risk reproducing inequalities.

Keywords: content moderation, NLP, Stuart Hall, Mary Douglas, pollution behavior, dirt, toxicity, racism, datafication