Profiling As Inferred Data. Amplifier Effects and Positive Feedback Loops

In: Emre Bayamlıoğlu, Irina Baraluic, Liisa Janssens and Mireille Hildebrandt (eds), BEING PROFILED:COGITAS ERGO SUM. 10 Years of Profiling the European Citizen, 2018: Amsterdam University Press., 112-115. DOI 10.5117/9789463722124/CH19.

4 Pages Posted: 18 Oct 2019

See all articles by Bart Custers

Bart Custers

Leiden University - Center for Law and Digital Technologies

Date Written: October 9, 2018

Abstract

Extracting profiles and other hidden knowledge from large amounts of data via techniques like data mining and machine learning is often regarded as an input-output process in which knowledge (i.e., profiles) are extracted from raw data. In this provocation, a different perspective is taken, in which profiles are not regarded as knowledge, but rather as (new) data, namely as inferred data. Using this perspective, it is shown that profiles are not only an end result or an end product, but can also be reused as ingredients for further data analytics. However, in this way, profiling processes may function as amplifiers, amplifying bias and inaccuracies via positive feedback loops, that further entrench consequences for data subjects.

Effects of small disturbances (like incorrect or incomplete data, or flaws in the data analysis) may lead to an increase of the magnitude of perturbations. Obviously, this may have some serious consequences for data subjects. Profiling based on datasets from data brokers that contain large amounts of inferred data, may propagate any existing biased patterns, leading, for instance, to disparate impact. The reuse of inferred data may also lead to self-fulfilling prophecies – a phenomenon well-known in profiling. In case of inferred data, however, the effect might be much stronger: because of the self-reinforcing effect, patterns may be amplified and become much more entrenched. These effects may amplify inequality, undermine democracy and further push people into categories that are hard to break out.

From a legal perspective, under the EU General Data Protection Regulation (GDPR) inferred data may or may not be personal data. If so, people have a right to access the inferred data and to receive meaningful information about the logic involved in the data analytics. However, since data subjects have no right to access the algorithms and data of other data subject used in the analyses, it is impossible for them to check whether data is inferred correctly.

Keywords: profiling, inferred data, data mining, GDPR, algorithms

Suggested Citation

Custers, Bart, Profiling As Inferred Data. Amplifier Effects and Positive Feedback Loops (October 9, 2018). In: Emre Bayamlıoğlu, Irina Baraluic, Liisa Janssens and Mireille Hildebrandt (eds), BEING PROFILED:COGITAS ERGO SUM. 10 Years of Profiling the European Citizen, 2018: Amsterdam University Press., 112-115. DOI 10.5117/9789463722124/CH19., Available at SSRN: https://ssrn.com/abstract=3466857 or http://dx.doi.org/10.2139/ssrn.3466857

Bart Custers (Contact Author)

Leiden University - Center for Law and Digital Technologies ( email )

2300 RA Leiden, NL-2300RA
Netherlands

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
368
Abstract Views
1,749
Rank
140,616
PlumX Metrics