Cdun: Co-Dual Unfolding Network for Multispectral and Panchromatic Images Fusion

Abstract

Pansharpening aims to acquire a high-resolution multispectral image (HRMS) by fusing a low-resolution MS image (LRMS) and a high-resolution panchromatic image (PAN). Existing Convolutional Neural Networks (CNNs)-based methods lack interpretability due to their black-box structure. Although current deep unfolding methods model the relationship between HRMS and observed images, their performance and generalization ability are limited due to the lack of more explicit intrinsic relationships between HRMS and observed images.To address these issues, this paper proposes a novel Co-Dual Unfolding Network (CDUN) for Pansharpening by combining the Retinex model and the degradation-aware model. We first use the Retinex model for LRMS as the observation model and design a Reflectance Learning sub-Net (RLN) to capture the reflectance shared with HRMS. Subsequently, we obtain the illumination of the HRMS through a Degradation-aware Illumination estimation sub-Net (DIN) with fixed reflectance. Finally, we utilize Retinex reconstruction loss and illumination-related loss to complete the end-to-end training of CDUN. Extensive experiments demonstrate the superiority of the proposed model over state-of-the-art methods and the effectiveness of the two sub-networks.

Keywords: remote sensing, Multi-Spectral image fusion, Deep unfolding network, Pansharpening, Retinex model