Algorithmic Decoupling and the Adoption of `Privacy-Preserving' Analytics

Abstract

Algorithmic techniques for privacy-preserving analytics (PPA) offer organizations a way to maintain and expand access to valuable data while preserving individuals’ privacy. Adoption of PPA techniques is growing in industry and government. However, their impacts on privacy protection are not yet clear. Small differences in design can have significant downstream consequences for data privacy, but little research has yet examined the motivations and decision-making processes behind PPA adoption, and how those decisions ultimately influence whether adoption lives up to consumers’ and regulators’ expectations. We investigate the organizational processes driving adoption and deployment of PPA systems in a qualitative study based on interviews with executives, lawyers, engineers, and scientists implementing PPA across large technology firms, startups, non-profits, and government agencies. We develop grounded theory to describe the processes by which organizations interpret social expectations into algorithmic designs and justify their design choices. We find several mechanisms by which organizations decouple representations about privacy from the specifics of their implementation with algorithms. We introduce a new dimension of organizational decoupling, algorithmic decoupling, to describe the ways algorithmic systems contribute to gaps between organizational policy, practice, and outcomes. We explore the consequences of algorithmic decoupling for the future of regulation and research related to privacy and other issues.

Keywords: privacy-preserving technology, technology adoption, organizational decoupling, legal endogeneity, algorithmic systems