High-Quality Codebook Prior Guided Pansharpening

Abstract

Pansharpening aims to enhance multispectral (MS) imagery by integrating high-resolution spatial details from panchromatic (PAN) data while preserving spectral fidelity, thereby generating high-resolution MS (HRMS) images critical for advanced remote sensing applications. Despite deep learning (DL) becoming the dominant paradigm for this task, current methods inadequately exploit the inherent physical priors of HRMS images during training, leading to suboptimal equilibrium between spatial enhancement and spectral preservation. To bridge this gap, we propose \textbf{High-Quality Prior guided Information Augmentation Network} (HQPIA-Net), a novel codebook learning framework that systematically leverages high-quality (HQ) spectral-spatial priors from HRMS references. Our method implements a three-stage training paradigm: (1) self-supervised codebook initialization through HRMS reconstruction, (2) cross-modal alignment of PAN and low-quality (LQ) MS features via a Feature Fusion Module (FFM), and (3) hierarchical enhancement using a Multi-Level Information Augmentation Module (MLIAM) that combines feature-level attention mechanisms (DIAM) and image-level adaptive filtering (SIAM) to prevent information loss during quantization. Besides, a channel information-aware loss dynamically prioritizes spectral bands based on dataset-level entropy analysis, ensuring balanced codebook representations. Comprehensive evaluations across two intra-satellite and one cross-satellite dataset demonstrate state-of-the-art performance in both reduced- and full-resolution scenarios, with particular robustness against input distribution shifts.

Keywords: Pansharpening, Codebook, Feature quantization