Building a Culture of Safety for AI: Perspectives and Challenges

Abstract

Artificial Intelligence (AI) shows great promise, but poses great risks - including problems with fairness, accountability, and transparency, as well as far larger emerging and/or existential risks. These latter risks have been labeled a “global priority,” and concrete steps for addressing them must be taken. One proposed step is building a culture of safety in the industry for developing and deploying this technology. This paper explores challenges, based on experiences of high-risk industries, and how they do and do not apply.

The concept of safety culture has a long history in domains like heavy industry, nuclear power, and healthcare, which emphasize preventive measures, standardized procedures, and a strong commitment to safety. Building a safety culture for AI faces different challenges; damage is often intangible and remote, affecting users, not developers, and rapid advances introduce emergent and unforeseen risks. The creation of a safety culture is also difficult due to the lack of consensus on risk prioritization, a business and research operating environment which ignores the risks, and a lack of standardized safety practices. Moreover, cultural change is difficult to impose, and current culture is incompatible with safety.

To overcome these challenges, a comprehensive strategy would identify, anticipate, and address risks, proactively use red-teams, and audit to evaluate safety - but these are insufficient in isolation, or without a safety culture. Those developing and deploying AI must have a flexible and forward-looking risk culture, while explicitly prioritizing safety over profitability.

The challenges are serious, but not inimical to an AI safety culture. Moving forward, an inclusive dialogue with industry, regulators, and academic experts is critical. At the same time, the delay tactic of waiting for consensus can be fatal, and all parties must immediately start building a culture able to confront and address the risks posed by AI systems.

Keywords: Safety Culture, Artificial Intelligence, Organizational Culture, Risks of Artificial Intelligence