Content Moderation as Surveillance

Abstract

Technology platforms are the new governments, and content moderation is the new law, or so goes a common refrain. And as platforms increasingly turn toward new, automated mechanisms of enforcing their rules, the apparent power of the private sector seems only to grow. Yet beneath the surface lies a web of complex relationships between public and private authorities that call into question whether platforms truly possess such unilateral power. Law enforcement and police are exerting influence over platform content rules, giving governments a louder voice in supposedly “private” decisions. At the same time, law enforcement avails itself of the affordances of social media in detecting, investigating, and preventing crime.

This Article, prepared for a symposium dedicated to Joel Reidenberg’s germinal article Lex Informatica, untangles the relationship between content moderation and surveillance. Building on Reidenberg’s fundamental insights regarding the relationships between rules imposed by legal regimes and those imposed by technological design, the Article first traces how content moderation rules intersect with law enforcement, including through formal demands for information, informal relationships between platforms and law enforcement agencies, and the impact of end-to-end encryption. Second, it critically assesses the degree to which government involvement in content moderation actually tempers platform power. Rather than effective oversight and checking of private power, it contends, the emergent arrangements between platforms and law enforcement institutions foster mutual embeddedness and the entrenchment of private authority within public governance.

Keywords: content moderation, free speech, privacy, surveillance, private platforms, internet platforms, internet intermediaries, private governance, artificial intelligence, automation, algorithmic governance

JEL Classification: K14