Beyond Automation: Machine Learning-Based Systems and Human Behavior in the Personalization Economy

52 Pages Posted: 30 Aug 2021

See all articles by Niklas Eder

Niklas Eder

Information Society Project - Yale Law School; Humboldt University, Law School; University of Leipzig, Law School

Date Written: August 3, 2021


Corporations seek to exploit the enormous economic potential of personalization beyond the confinements of the online space. They increasingly rely on machine learning-based systems to select and evaluate their customers. These systems define how we are perceived and tailor the way in which are treated. The implications of these systems are already immense, and they foreshadow a larger transformation. Over the course of the 21st century, ubiquitous, machine learning-based personalization will become a fundamental condition of human existence, shaping our behavior and constraining our freedom.

Legal scholarship needs a conceptual foundation to address urgent questions about how personalization, driven by machine learning-based decision-making, affects liberty and other liberal democratic values. Drawing on surveillance theory, this article develops that foundation and examines the normative and constitutional implications of ubiquitous personalization. The article builds on the concepts of panopticism and the surveillance assemblage to analyze how machine learning-based decision-making amplifies corporate power and governs human behavior. It argues that existing legal responses, focusing on rights, explainability and transparency, fail to prevent the already fragile balance of power between individuals and corporations from tipping in corporations’ interest. To adequately respond to machine learning-based decision-making, legal scholarship must overcome its individualistic focus and engage in a debate on the legitimacy of corporate surveillance.

Keywords: Big Data, AI, Automated Decision-Making, Corporate Surveillance, Panopticism, Surveillance Assemblage, Legitimacy, machine-learning, transparency, fundamental rights

Suggested Citation

Eder, Niklas, Beyond Automation: Machine Learning-Based Systems and Human Behavior in the Personalization Economy (August 3, 2021). Stanford Technology Law Review, Forthcoming, Available at SSRN:

Niklas Eder (Contact Author)

Information Society Project - Yale Law School ( email )

127 Wall Street
New Haven, CT 06511
United States

Humboldt University, Law School ( email )

Unter den Linden 6
Berlin, D-10099

University of Leipzig, Law School ( email )

Otto-Schill-Str. 2
04109 Leipzig

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics