Risk Identification Questionnaire for Detecting Unintended Bias in the Machine Learning Development Lifecycle

Lee, Michelle Seng Ah and Jatinder Singh. "Risk Identification Questionnaire for Detecting Unintended Bias in the Machine Learning Development Lifecycle." Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 2021.

11 Pages Posted: 16 Feb 2021 Last revised: 24 May 2021

See all articles by Michelle Seng Ah Lee

Michelle Seng Ah Lee

University of Cambridge

Jatinder Singh

University of Cambridge -- Dept. Computer Science & Technology (Computer Laboratory)

Date Written: May 21, 2021

Abstract

Unintended biases in machine learning (ML) models have the potential to introduce undue discrimination and exacerbate social inequalities. The research community has proposed various technical and qualitative methods intended to assist practitioners in assessing these biases. While frameworks for identifying risks of harm due to unintended biases have been proposed, they have not yet been operationalised into practical tools to assist industry practitioners.

In this paper, we link prior work on bias assessment methods to phases of a standard organisational risk management process (RMP), noting a gap in measures for helping practitioners identify bias-related risks. Targeting this gap, we introduce a bias identification methodology and questionnaire, illustrating its application through a real-world practitioner-led use case. We validate the need and usefulness of the questionnaire through a survey of industry practitioners, which provides insights into their practical requirements and preferences. Our results indicate that such an approach is helpful for proactively uncovering unexpected bias concerns, particularly where it is easy to integrate into existing processes, and facilitates communication with non-technical stakeholders.

Ultimately, effective end-to-end management of ML risks requires a more targeted identification of potential harm and its sources, so that appropriate mitigation strategies can be formulated. Towards this, our questionnaire provides a practical means to assist practitioners in identifying bias-related risks.

Keywords: algorithmic fairness, algorithmic bias, unintended bias, machine learning, bias identification, risk identification, risk management, bias questionnaire, risk methods

Suggested Citation

Lee, Michelle Seng Ah and Singh, Jatinder, Risk Identification Questionnaire for Detecting Unintended Bias in the Machine Learning Development Lifecycle (May 21, 2021). Lee, Michelle Seng Ah and Jatinder Singh. "Risk Identification Questionnaire for Detecting Unintended Bias in the Machine Learning Development Lifecycle." Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 2021., Available at SSRN: https://ssrn.com/abstract=3777093 or http://dx.doi.org/10.2139/ssrn.3777093

Michelle Seng Ah Lee (Contact Author)

University of Cambridge ( email )

15 JJ Thomson Ave
William Gates Building
Cambridge, Cambridgeshire CB3 0FD
United Kingdom

HOME PAGE: http://compacctsys.soc.srcf.net/team/

Jatinder Singh

University of Cambridge -- Dept. Computer Science & Technology (Computer Laboratory) ( email )

15 JJ Thomson Avenue
William Gates Building
Cambridge, CB3 0FD
United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
130
Abstract Views
678
rank
265,350
PlumX Metrics