Robustness Assessment of Hyperspectral Image Cnns Using Metamorphic Testing

35 Pages Posted: 7 May 2022

See all articles by Rached Bouchoucha

Rached Bouchoucha

affiliation not provided to SSRN

Houssem Ben Braiek

affiliation not provided to SSRN

Foutse Khomh

affiliation not provided to SSRN

Sonia Bouzidi

affiliation not provided to SSRN

Rania Zaatour

affiliation not provided to SSRN

Abstract

Remote sensing has proven its utility in many critical domains, such as medicine, military, and ecology. Recently, we have been witnessing a surge in the adoption of deep learning (DL) techniques by the remote sensing community. DL-based classifiers, such as convolutional neural networks (CNNs), have been reported to achieve impressive predictive performances reaching 99\% of accuracy when applied to hyperspectral images (HSIs), a high-dimensional type of remote sensing data. However, these deep learners are known to be highly sensitive to even slight perturbations of their high-dimensional raw inputs. In real-world contexts, concerns can be raised about how robust they really are against corner-case scenarios. When HSI classifiers are applied in safety-critical applications, ensuring an adequate level of robustness is crucial to prevent unexpected system behaviors. Yet, there are few studies dealing with their robustness, nor are RGB-testing methods able to cover the HSI-specific challenges. This led us to propose a systematic testing method to assess the robustness of the CNNs trained to classify HSIs. First, we elaborate domain-specific metamorphic transformations that simulate naturally-occurring distortions of remote sensing HSIs. Then, we leverage metaheuristic search algorithms to optimize the fitness of synthetically-distorted inputs to stress the weaknesses of the on-testing CNN, while remaining in compliance with domain expert requirements, in order to preserve the semantic of the generated inputs. Relying on our metamorphic testing method, we assess the robustness of established and novel CNNs for HSI classification, and demonstrate their failure, on average, in 25\% of the produced test cases. Furthermore, we fine-tuned the tested CNNs on training data augmented with these failure-revealing metamorphic transformations. Results show that the fined-tuning successfully fixed at least 90\% of the CNN weaknesses, with less than 1\% of degradation in the original predictive performance, outperforming the common iterative gradient-based adversarial attack, namely, Projected Gradient Descent (PGD).

Keywords: Deep Learning Testing, Metamorphic testing, Software Testing, Remote Sensing, Hyperspectral Image Classification

Suggested Citation

Bouchoucha, Rached and Ben Braiek, Houssem and Khomh, Foutse and Bouzidi, Sonia and Zaatour, Rania, Robustness Assessment of Hyperspectral Image Cnns Using Metamorphic Testing. Available at SSRN: https://ssrn.com/abstract=4102952 or http://dx.doi.org/10.2139/ssrn.4102952

Rached Bouchoucha (Contact Author)

affiliation not provided to SSRN ( email )

No Address Available

Houssem Ben Braiek

affiliation not provided to SSRN ( email )

No Address Available

Foutse Khomh

affiliation not provided to SSRN ( email )

No Address Available

Sonia Bouzidi

affiliation not provided to SSRN ( email )

No Address Available

Rania Zaatour

affiliation not provided to SSRN ( email )

No Address Available

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
63
Abstract Views
255
Rank
634,810
PlumX Metrics