Can Racial Bias in Policing Be Credibly Estimated Using Data Contaminated by Post-Treatment Selection?

38 Pages Posted: 12 Oct 2021

See all articles by Dean Knox

Dean Knox

The Wharton School of the University of Pennsylvania

Will Lowe

Princeton University - Department of Political Science

Jonathan Mummolo

Princeton University

Date Written: September 11, 2020

Abstract

Studies of racial bias in policing often rely on data contaminated by selection issues, e.g. using records of stops or arrests—which may themselves be a product of racial bias—to estimate discrimination in subsequent actions like use of force. This feature raises the threat of post-treatment-selection bias, which recent work shows can lead to severe underestimates of discrimination. However, prominent studies continue to ignore this issue, employing standard regression techniques with contaminated data. In this paper, we formally analyze the key identifying assumption undergirding these studies, “subset ignorability,” and show it corresponds to the measure-zero set of knife-edge conditions in which differing biases happen to sum to zero. Because there is no substantive reason to believe such accidental cancellation would occur, we conclude this approach is not reliable in applied research, and we emphasize the need for continued caution and increased rigor in high-stakes analyses of discriminatory policing with contaminated data.

Suggested Citation

Knox, Dean and Lowe, Will and Mummolo, Jonathan, Can Racial Bias in Policing Be Credibly Estimated Using Data Contaminated by Post-Treatment Selection? (September 11, 2020). Available at SSRN: https://ssrn.com/abstract=3940802 or http://dx.doi.org/10.2139/ssrn.3940802

Dean Knox (Contact Author)

The Wharton School of the University of Pennsylvania ( email )

Philadelphia, PA 19104
United States

Will Lowe

Princeton University - Department of Political Science

Fisher Hall
Princeton, NJ 08544-1012
United States

Jonathan Mummolo

Princeton University ( email )

22 Chambers Street
Princeton, NJ 08544-0708
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
115
Abstract Views
521
Rank
433,300
PlumX Metrics