Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles

27 Pages Posted: 22 Mar 2021

See all articles by Michael Merz

Michael Merz

University of Hamburg

Ronald Richman

Old Mutual Insure; University of the Witwatersrand

Andreas Tsanakas

Bayes Business School (formerly Cass), City, University of London

Mario V. Wuthrich

RiskLab, ETH Zurich

Date Written: March 22, 2021

Abstract

A vastly growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specificalllly, we consider variable importance by mixing (global) output levels and, thus, explain how features marginally contribute across different regions of the prediction space. Hence, MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects (ALE), which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effect, and visually illustrate the 3-way relationship between marginal attribution, output level, and feature value.

Keywords: explainable AI (XAI), model-agnostic tools, deep learning, attribution, accumulated local e ects (ALE), partial dependence plot (PDP), locally interpretable model-agnostic explanation (LIME), variable importance, post-hoc analysis

JEL Classification: C02

Suggested Citation

Merz, Michael and Richman, Ronald and Tsanakas, Andreas and Wuthrich, Mario V., Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles (March 22, 2021). Available at SSRN: https://ssrn.com/abstract=3809674 or http://dx.doi.org/10.2139/ssrn.3809674

Michael Merz

University of Hamburg ( email )

Allende-Platz 1
Hamburg, 20146
Germany

Ronald Richman

Old Mutual Insure ( email )

Wanooka Place
St Andrews Road
Johannesburg, 2192
South Africa

University of the Witwatersrand ( email )

1 Jan Smuts Avenue
Johannesburg, GA Gauteng 2000
South Africa

Andreas Tsanakas

Bayes Business School (formerly Cass), City, University of London ( email )

106 Bunhill Row
London, EC1Y 8TZ
United Kingdom

Mario V. Wuthrich (Contact Author)

RiskLab, ETH Zurich ( email )

Department of Mathematics
Ramistrasse 101
Zurich, 8092
Switzerland

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
271
Abstract Views
882
rank
154,474
PlumX Metrics