Analyzing Privacy Policies Using Contextual Integrity Annotations

18 Pages Posted: 1 Oct 2018

See all articles by Yan Shvartzshnaider

Yan Shvartzshnaider

New York University (NYU); Princeton University

Noah Apthorpe

Colgate University

Nick Feamster

University of Chicago

Helen Nissenbaum

New York University; Cornell Tech

Date Written: September 5, 2018


In this paper, we demonstrate the effectiveness of using the theory of contextual integrity (CI) to annotate and evaluate privacy policy statements. We perform a case study using CI annotations to compare Facebook's privacy policy before and after the Cambridge Analytica scandal. The updated Facebook privacy policy provides additional details about what information is being transferred, from whom, by whom, to whom, and under what conditions. However, some privacy statements prescribe an incomprehensibly large number of information flows by including many CI parameters in single statements. Other statements result in incomplete information flows due to the use of vague terms or omitting contextual parameters altogether. We then demonstrate that crowdsourcing can effectively produce CI annotations of privacy policies at scale. We test the CI annotation task on 48 excerpts of privacy policies from 17 companies with 141 crowdworkers. The resulting high precision annotations indicate that crowdsourcing could be used to produce a large corpus of annotated privacy policies for future research.

Keywords: Privacy policies, contextual integrity, annotation

Suggested Citation

Shvartzshnaider, Yan and Apthorpe, Noah and Feamster, Nick and Nissenbaum, Helen F. and Nissenbaum, Helen F., Analyzing Privacy Policies Using Contextual Integrity Annotations (September 5, 2018). Available at SSRN: or

Yan Shvartzshnaider (Contact Author)

New York University (NYU) ( email )

Bobst Library, E-resource Acquisitions
20 Cooper Square 3rd Floor
New York, NY 10003-711
United States

Princeton University ( email )

22 Chambers Street
Princeton, NJ 08544-0708
United States

Noah Apthorpe

Colgate University ( email )

13 Oak Drive
Hamilton NY 13346, NY 13346
United States

Nick Feamster

University of Chicago ( email )

Chicago, IL
United States

Helen F. Nissenbaum

Cornell Tech ( email )

111 8th Avenue #302
New York, NY 10011
United States

New York University ( email )

New York, NY 10003
United States
212-998-5251 (Phone)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics