Analyzing Privacy Policies Using Contextual Integrity Annotations

18 Pages Posted: 1 Oct 2018

See all articles by Yan Shvartzshnaider

Yan Shvartzshnaider

New York University (NYU); Princeton University

Noah Apthorpe

Princeton University

Nick Feamster

University of Chicago

Helen Nissenbaum

Cornell Tech; New York University

Date Written: September 5, 2018


In this paper, we demonstrate the effectiveness of using the theory of contextual integrity (CI) to annotate and evaluate privacy policy statements. We perform a case study using CI annotations to compare Facebook's privacy policy before and after the Cambridge Analytica scandal. The updated Facebook privacy policy provides additional details about what information is being transferred, from whom, by whom, to whom, and under what conditions. However, some privacy statements prescribe an incomprehensibly large number of information flows by including many CI parameters in single statements. Other statements result in incomplete information flows due to the use of vague terms or omitting contextual parameters altogether. We then demonstrate that crowdsourcing can effectively produce CI annotations of privacy policies at scale. We test the CI annotation task on 48 excerpts of privacy policies from 17 companies with 141 crowdworkers. The resulting high precision annotations indicate that crowdsourcing could be used to produce a large corpus of annotated privacy policies for future research.

Keywords: Privacy policies, contextual integrity, annotation

Suggested Citation

Shvartzshnaider, Yan and Apthorpe, Noah and Feamster, Nick and Nissenbaum, Helen F., Analyzing Privacy Policies Using Contextual Integrity Annotations (September 5, 2018). Available at SSRN: or

Yan Shvartzshnaider (Contact Author)

New York University (NYU) ( email )

Bobst Library, E-resource Acquisitions
20 Cooper Square 3rd Floor
New York, NY 10003-711
United States

Princeton University ( email )

22 Chambers Street
Princeton, NJ 08544-0708
United States

Noah Apthorpe

Princeton University ( email )

35 Olden Street
Princeton, NJ 08540
United States

Nick Feamster

University of Chicago ( email )

Chicago, IL
United States

Helen F. Nissenbaum

Cornell Tech ( email )

111 8th Avenue #302
New York, NY 10011
United States

New York University ( email )

New York, NY 10003
United States
212-998-5251 (Phone)

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
PlumX Metrics