Download this Paper Open PDF in Browser

Disagreeable Privacy Policies: Mismatches between Meaning and Users’ Understanding

118 Pages Posted: 31 Mar 2014 Last revised: 15 Oct 2015

Joel R. Reidenberg

Fordham University School of Law

Travis Breaux

Carnegie Mellon University

Lorrie Faith Cranor

Carnegie Mellon University - School of Computer Science and Carnegie Institute of Technology

Brian French

Institute for Software Research, Carnegie Mellon

Amanda Grannis

Fordham Center on Law and Information Policy (CLIP)

James T. Graves

Carnegie Mellon University

Fei Liu

Carnegie Mellon University

Aleecia M. McDonald

Stanford University

Thomas B. Norton

Fordham University - Fordham Center on Law and Information Policy (CLIP)

Rohan Ramanath

Carnegie Mellon University

N. Cameron Russell

Fordham Center on Law and Information Policy (CLIP)

Norman Sadeh

Carnegie Mellon University - School of Computer Science

Florian Schaub

University of Michigan at Ann Arbor - School of Information

Date Written: August 15, 2014

Abstract

Privacy policies are verbose, difficult to understand, take too long to read, and may be the least-read items on most websites even as users express growing concerns about information collection practices. For all their faults, though, privacy policies remain the single most important source of information for users to attempt to learn how companies collect, use, and share data. Likewise, these policies form the basis for the self-regulatory notice and choice framework that is designed and promoted as a replacement for regulation. The underlying value and legitimacy of notice and choice depends, however, on the ability of users to understand privacy policies.

This paper investigates the differences in interpretation among expert, knowledgeable, and typical users and explores whether those groups can understand the practices described in privacy policies at a level sufficient to support rational decision-making. The paper seeks to fill an important gap in the understanding of privacy policies through primary research on user interpretation and to inform the development of technologies combining natural language processing, machine learning and crowdsourcing for policy interpretation and summarization.

For this research, we recruited a group of law and public policy graduate students at Fordham University, Carnegie Mellon University, and the University of Pittsburgh (“knowledgeable users”) and presented these law and policy researchers with a set of privacy policies from companies in the e-commerce and news & entertainment industries. We asked them nine basic questions about the policies’ statements regarding data collection, data use, and retention. We then presented the same set of policies to a group of privacy experts and to a group of non-expert users.

The findings show areas of common understanding across all groups for certain data collection and deletion practices, but also demonstrate very important discrepancies in the interpretation of privacy policy language, particularly with respect to data sharing. The discordant interpretations arose both within groups and between the experts and the two other groups.

The presence of these significant discrepancies has critical implications. First, the common understandings of some attributes of described data practices mean that semi-automated extraction of meaning from website privacy policies may be able to assist typical users and improve the effectiveness of notice by conveying the true meaning to users. However, the disagreements among experts and disagreement between experts and the other groups reflect that ambiguous wording in typical privacy policies undermines the ability of privacy policies to effectively convey notice of data practices to the general public.

The results of this research will, consequently, have significant policy implications for the construction of the notice and choice framework and for the US reliance on this approach. The gap in interpretation indicates that privacy policies may be misleading the general public and that those policies could be considered legally unfair and deceptive. And, where websites are not effectively conveying privacy policies to consumers in a way that a “reasonable person” could, in fact, understand the policies, “notice and choice” fails as a framework. Such a failure has broad international implications since websites extend their reach beyond the United States.

Notes: Funding for this project was in part provided by the National Science Foundation under its Secure and Trustworthy Computing (SaTC) initiative grants 1330596, 1330214, and 1330141 for “TWC SBE: Option: Frontier: Collaborative: Towards Effective Web Privacy Notice and Choice: A Multi-Disciplinary Prospective.”

Keywords: privacy policies, natural language processing, automated processing, notice and choice,

Suggested Citation

Reidenberg, Joel R. and Breaux, Travis and Cranor, Lorrie Faith and French, Brian and Grannis, Amanda and Graves, James T. and Liu, Fei and McDonald, Aleecia M. and Norton, Thomas B. and Ramanath, Rohan and Russell, N. Cameron and Sadeh, Norman and Schaub, Florian, Disagreeable Privacy Policies: Mismatches between Meaning and Users’ Understanding (August 15, 2014). Fordham Law Legal Studies Research Paper No. 2418297; 2014 TPRC Conference Paper; Berkeley Technology Law Journal, Vol. 30, 2015; Fordham Law Legal Studies Research Paper No. 2418297. Available at SSRN: https://ssrn.com/abstract=2418297

Joel R. Reidenberg (Contact Author)

Fordham University School of Law ( email )

140 West 62nd Street
New York, NY 10023
United States
212-636-6843 (Phone)
212-930-8833 (Fax)

HOME PAGE: http://faculty.fordham.edu/reidenberg

Travis Breaux

Carnegie Mellon University ( email )

Pittsburgh, PA 15213-3890
United States

Lorrie Faith Cranor

Carnegie Mellon University - School of Computer Science and Carnegie Institute of Technology ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213
United States

Brian French

Institute for Software Research, Carnegie Mellon ( email )

United States

Amanda Grannis

Fordham Center on Law and Information Policy (CLIP) ( email )

Fordham Law School
140 West 62nd Street
New York, NY 10023
United States

James T. Graves

Carnegie Mellon University ( email )

Pittsburgh, PA 15213-3890
United States

Fei Liu

Carnegie Mellon University ( email )

Pittsburgh, PA 15213-3890
United States

Aleecia M. McDonald

Stanford University ( email )

Stanford, CA 94305
United States

Thomas B. Norton

Fordham University - Fordham Center on Law and Information Policy (CLIP) ( email )

New York, NY
United States

Rohan Ramanath

Carnegie Mellon University ( email )

Pittsburgh, PA 15213-3890
United States

N. Cameron Russell

Fordham Center on Law and Information Policy (CLIP) ( email )

Fordham Law School
140 West 62nd Street
New York, NY 10023
United States
212-930-8878 (Phone)

Norman Sadeh

Carnegie Mellon University - School of Computer Science ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213
United States

Florian Schaub

University of Michigan at Ann Arbor - School of Information ( email )

105 S State St
Ann Arbor, MI 48109
United States

Paper statistics

Downloads
582
Rank
37,455
Abstract Views
2,420