Content-Oblivious Trust and Safety Techniques: Results from a Survey of Online Service Providers

Pfefferkorn, R. (2022). Content-Oblivious Trust and Safety Techniques: Results from a Survey of Online Service Providers. Journal of Online Trust and Safety, 1(2). https://doi.org/10.54501/jots.v1i2.14

38 Pages Posted: 10 Sep 2021 Last revised: 2 Mar 2022

Date Written: February 28, 2022

Abstract

We present the results of a survey about the trust and safety techniques of a group of online service providers that collectively serve billions of users. We classify techniques that require the provider to be able to access the contents of users’ files and communications at will as content dependent, and content oblivious otherwise. We find that more providers use abuse-reporting features (which are content oblivious) than other abuse-detection techniques, but that participants’ abuse-reporting tools do not consistently cover the types of abuse that users may en- counter. We also find that, despite strong consensus among participating providers that automated content scanning (which is content dependent) is the most useful means of detecting child sex abuse imagery, they do not consider it to be nearly as useful for other kinds of abuse. These results indicate that content-dependent techniques do not constitute a silver bullet to protect users against abuse. They also demonstrate that the impact of end-to-end encryption (which, controversially, impedes outside access to user content) on abuse detection may vary by abuse type. These findings have implications for policy debates over the regulation of online service providers’ anti-abuse obligations and their use of end-to-end encryption.

Keywords: end-to-end encryption, encryption, online abuse, online safety, trust and safety

Suggested Citation

Pfefferkorn, Riana, Content-Oblivious Trust and Safety Techniques: Results from a Survey of Online Service Providers (February 28, 2022). Pfefferkorn, R. (2022). Content-Oblivious Trust and Safety Techniques: Results from a Survey of Online Service Providers. Journal of Online Trust and Safety, 1(2). https://doi.org/10.54501/jots.v1i2.14, Available at SSRN: https://ssrn.com/abstract=3920031 or http://dx.doi.org/10.2139/ssrn.3920031

Riana Pfefferkorn (Contact Author)

Stanford Internet Observatory ( email )

Stanford, CA 94305
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,197
Abstract Views
6,930
Rank
30,701
PlumX Metrics