Case Study: The Distilling of a Biased Algorithmic Decision System through a Business Lens

35 Pages Posted: 26 Mar 2022

See all articles by Merve Hickok

Merve Hickok

AIethicist.org

Colleen Dorsey

University of St Thomas

Tim O'Brien

Master of Jurisprudence

Dorothea Baur

Baur Consulting

Katrina Ingram

University of Alberta

Chhavi Chauhan

The American Society for Investigative Pathology

Attlee Gamundani

United Nations University Institute in Macau; Namibia University of Science and Technology

Date Written: January 28, 2022

Abstract

Technological advances embedded within algorithmic decision systems are being deployed every day. Some of these investments are unquestionably worthwhile, while some others prioritize commercialization of technology ahead of societal impact. The article uses a real-world case from the healthcare sector to demonstrate the design and governance shortfalls of an algorithmic tool through its lifecycle. The healthcare sector sits on a mine of data, making it one of the most lucrative fields for big data–based analytics. However, the remunerative healthcare sector is particularly sensitive to the quality of data and algorithmic design decisions, making it paramount for all stakeholders to safely and ethically develop, deploy and implement the algorithmic tools. Otherwise, these systems can have detrimental effects on the life, well-being, and safety of patients. The authors provide guidance on responsible and sustainable deployment practices applicable across industries. The systematic dissection of this case can be applied to different systems across different domains like employment, credit scoring, housing, education, criminal justice, and many others. Every business automating and streamlining processes and core functions through digital transformation needs to implement new accountability and governance mechanisms and invest in the inevitable culture change necessary.

Keywords: algorithmic decision, artificial intelligence, automated decision, bias, discrimination, ethics, accountability, algorithm, governance

JEL Classification: I14, I18, M14, M48, Z18

Suggested Citation

Hickok, Merve and Dorsey, Colleen and O'Brien, Tim and Baur, Dorothea and Ingram, Katrina and Chauhan, Chhavi and Gamundani, Attlee, Case Study: The Distilling of a Biased Algorithmic Decision System through a Business Lens (January 28, 2022). Available at SSRN: https://ssrn.com/abstract=4019672 or http://dx.doi.org/10.2139/ssrn.4019672

Merve Hickok (Contact Author)

AIethicist.org ( email )

United States

Colleen Dorsey

University of St Thomas ( email )

Tim O'Brien

Master of Jurisprudence ( email )

William H. Gates Hall
Box 353020
Seattle, WA 98105-3020
United States

Dorothea Baur

Baur Consulting ( email )

United States

Katrina Ingram

University of Alberta ( email )

Edmonton, Alberta T6G 2R3
Canada

Chhavi Chauhan

The American Society for Investigative Pathology ( email )

United States

Attlee Gamundani

United Nations University Institute in Macau ( email )

Macau

Namibia University of Science and Technology ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
410
Abstract Views
1,605
Rank
154,032
PlumX Metrics