The Siren Call of Content Moderation Formalism

NEW TECHNOLOGIES OF COMMUNICATION AND THE FIRST AMENDMENT: THE INTERNET, SOCIAL MEDIA AND CENSORSHIP (Lee Bollinger & Geoffrey Stone eds., 2022 Forthcoming)

24 Pages Posted: 12 Jan 2022

Date Written: January 10, 2022

Abstract

Systems of online content moderation governance are becoming some of the most elaborate and extensive bureaucracies in history, and they are deeply imperfect and need reform. Would-be reformers of content moderation systems are drawn to a highly rule-bound and formalistic vision of how these bureaucracies should operate, but the sprawling chaos of online speech is too vast, ever-changing, and varied to be brought into consistent compliance with rigid rules. This essay argues that the quest to make content moderation systems ever more formalistic will not fix public and regulatory concerns about the legitimacy and accountability of how platforms moderate content on their services. The largest social media platforms operate massive unelected, unaccountable, and increasingly complex bureaucracies that decide to act or not act on millions of pieces of content uploaded to their platforms every day. A formalistic model, invoking judicial-style norms of reasoning and precedent, is doomed to fail at this scale and complexity. As these governance systems mature, it is time to be content moderation realists about the task ahead.

Keywords: content moderation, formalism, realism, transparency, platform governance

Suggested Citation

Douek, Evelyn, The Siren Call of Content Moderation Formalism (January 10, 2022). NEW TECHNOLOGIES OF COMMUNICATION AND THE FIRST AMENDMENT: THE INTERNET, SOCIAL MEDIA AND CENSORSHIP (Lee Bollinger & Geoffrey Stone eds., 2022 Forthcoming), Available at SSRN: https://ssrn.com/abstract=4005314 or http://dx.doi.org/10.2139/ssrn.4005314

Evelyn Douek (Contact Author)

Stanford Law School ( email )

559 Nathan Abbott Way
Stanford, CA 94305
United States

HOME PAGE: http://https://law.stanford.edu/directory/evelyn-douek/

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
268
Abstract Views
704
rank
167,024
PlumX Metrics