Analysis of the Clever Hans Effect in COVID-19 Detection Using Chest X-Ray Images and Bayesian Deep Learning

29 Pages Posted: 12 Apr 2023

See all articles by Julián D. Arias-Londoño

Julián D. Arias-Londoño

Universidad Politécnica de Madrid - Department of Signals, Systems and Radiocommunications

Juan Ignacio Godino-Llorente

Universidad Politécnica de Madrid - Department of Signals, Systems and Radiocommunications

Abstract

In recent months, the detection of COVID-19 from radiological images has become a research topic of significant interest. This is because previous results demonstrate the feasibility of the application, but literature has also reported some biases of the systems developed, which significantly limit their translation to the clinic. This paper deals with a set of explainability/interpretability techniques to analise spurious correlations during the inference, the consistency of the decisions, the uncertainty of the models, their performance, and certain biasing effects. For this purpose, we used two different off-the-shelf convolutional neural networks (DenseNet-121 and EfficientNet-B6) and their Bayesian counterparts. In view of the results, DenseNet is preferred in both its deteministic and stochastic versions, reaching BAcc over 97 % training with a large dataset (more than 70,000 images). However, results demonstrate that the artificial models are significantly affected by the Clever Hans effect, which is minimized pre-processing with a semantic segmentation of the lungs to guide the learning process towards an area with a causal relationship with the problem under study. Conclusions could also be extrapolated to the general context of pneumonia detection from plain chest X-Ray images.

Note:
Funding Information: Funded by the agreement between Comunidad de Madrid (Consejerıa de Educacion, Universidades, Ciencia y Portavocıa) and Universidad Politecncia de Madrid, to finance research actions on SARS-CoV-2 and COVID-19 disease with the REACT-UE resources of the European Regional Development Funds. This work was also supported by the Ministry of Economy and Competitiveness of Spain under Grant DPI2017-83405-R1.

Declaration of Interests: None.

Ethics Approval Statement: This was a retrospective study using non-institutional public Health Insurance Portability and Accountability Act (HIPAA) compliant deidentified imaging data. Ethical review and approval were waived for this reason. Patient consent was waived due to the use of open data.

Keywords: Deep learning, Bayesian Learning, explainability, Uncertainty, Calibration, COVID-19, Pneumonia, Radiological Imaging, Chest X-Ray.

Suggested Citation

Arias, Julián and Godino-Llorente, Juan Ignacio, Analysis of the Clever Hans Effect in COVID-19 Detection Using Chest X-Ray Images and Bayesian Deep Learning. Available at SSRN: https://ssrn.com/abstract=4414659 or http://dx.doi.org/10.2139/ssrn.4414659

Julián Arias (Contact Author)

Universidad Politécnica de Madrid - Department of Signals, Systems and Radiocommunications

Juan Ignacio Godino-Llorente

Universidad Politécnica de Madrid - Department of Signals, Systems and Radiocommunications ( email )

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
76
Abstract Views
252
Rank
561,582
PlumX Metrics