What Firms Must Know Before Adopting AI: The Ethics of AI Transparency

Abstract

Firms have obligations to stakeholders that do not disappear when managers adopt AI decision systems. We introduce the concept of the AI knowledge gap – where AI provides limited information about its operations while the stakeholder demands for information justifying firm decisions increase. We develop a framework of what firms must know about their AI model in the procurement process to ensure they understand how the model allows a firm to meet existing obligations including the anticipated risks of using the AI decision system, how to prevent foreseeable risks, and have a plan for resilience. We argue there are no conditions where it is ethical to unquestioningly adopt recommendations from a black box AI program within an organization. According to this argument, adequate comprehension and knowledge about an AI model is not a negotiable design feature but a strategic and moral requirement.