Strategic Games and Algorithmic Secrecy
41 Pages Posted: 24 Aug 2019 Last revised: 16 Nov 2020
Date Written: October 18, 2019
We challenge a claim commonly made by industry and government representatives and echoed by legal scholarship: that algorithmic decision-making processes are better kept opaque or secret because otherwise decision-subjects will “game the system”, leading to inaccurate or unfair results. We show that the range of situations in which people are able to game decision-making algorithms is narrow, even when there is substantial disclosure. We then analyze how to identify when gaming is possible in light of (i) how tightly the decision-making proxies are tied to the factors that would ideally determine the outcome, (ii) how easily those proxies can be altered by decision-subjects, and (iii) whether such strategic alterations ultimately lead to mistaken decisions. Based on this analysis, we argue that blanket claims that disclosure will lead to gaming are over-blown and that it will often be possible to construct socially beneficial disclosure regimes.
Keywords: algorithmic decision-making, gaming, algorithmic transparency, algorithmic secrecy, algorithmic opacity, algorithmic accountability, automated decision-making, credit scoring, criminal investigation, criminal procedure, employment law game theory & the law
Suggested Citation: Suggested Citation