Safeaipackage: A Python Package for AI Risk Measurement

61 Pages Posted: 1 Apr 2024

See all articles by Golnoosh Babaei

Golnoosh Babaei

University of Pavia

Paolo Giudici

University of Pavia

Emanuela Raffinetti

University of Pavia

Date Written: March 1, 2024

Abstract

The growth of Artificial Intelligence (AI) applications requires to develop risk management models that can balance opportunities with risks. In this paper, we contribute to the the debate on regulations and industry standards for AI risk management models proposing a set of integrated safe AI statistical metrics and a safeaipackage soft- ware toolbox, an open-source Python package that can measure the Security, Accuracy, Fairness, Explainability and, more generally, the risks of compliance of any Artificial Intelligence application.
Our proposed metrics are consistent with each other, as they are all derived from a common underlying statistical methodology: the Lorenz curve. They are easy to interpret, as are all expressed in percentage of an ideal situation of full compliance. They are agnostic, as they can be applied to any machine learning methods, regardless of the underlying data and model. They are fully reproducible, by means of the proposed Python code. After an introductory background, we describe our methodological framework, named “Rank Graduation Box”, as it allows to derive any necessary compliance metric for AI by means of a pairwise com- parison of Lorenz curves, based on different rank graduations. In the paper, we specifically consider metrics for the assessment of Security, Accuracy, Fairness, Explainability and Privacy. We then describe the Python safeaipackage software, which allows the application of all the proposed metrics for the assessment of the regulatory compliance of any AI output. The software serves both as a user-ready toolbox that implements our formalized metrics for evaluating the compliance of AI models, and as a convenient development environment for Python programmers, which can advance the field of responsible AI by implementing further metrics directly in Python.

Keywords: Artificial Intelligence Risk Management, Concordance, Lorenz Curves, Ranks, Responsible Artificial Intelligence, Python

Suggested Citation

Babaei, Golnoosh and Giudici, Paolo and Raffinetti, Emanuela, Safeaipackage: A Python Package for AI Risk Measurement (March 1, 2024). Available at SSRN: https://ssrn.com/abstract=4744576 or http://dx.doi.org/10.2139/ssrn.4744576

Golnoosh Babaei (Contact Author)

University of Pavia ( email )

Via San Felice
5
Pavia, Pavia 27100
Italy

Paolo Giudici

University of Pavia ( email )

Via San Felice 7
27100 Pavia, 27100
Italy

Emanuela Raffinetti

University of Pavia ( email )

Via San Felice 5
Pavia, 27100
Italy

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
165
Abstract Views
567
Rank
338,879
PlumX Metrics