1 December 2025 in IEEE Transactions on Geoscience and Remote Sensing
Multimodal fusion widely uses convolutional layers to capture local correlations and adjust feature dimensions. However, the progressive expansion of the receptive field in convolutional layers often compromises spatial context retention, leading to the loss of fine details. Furthermore, the fixed-size kernels typically used in standard convolution restrict the network’s ability to capture multiscale contextual details. To address this limitation, this paper develops a dilated transformation-guided unsupervised multimodal learning (DTUML) method to fuse a high-resolution multispectral image (HR-MSI) and a low-resolution hyperspectral image (LR-HSI), thereby generating a high-resolution hyperspectral image (HR-HSI). Our DTUML adopts a dual-stream encoder architecture to conduct multimodal data, where one stream focuses on preserving spectral information from LR-HSIs, while the other emphasizes the acquisition of spatial details from HR-MSIs. These complementary features are subsequently integrated to ensure spectral fidelity and retain spatial detail. Then, a convolutional layer restores dimensional consistency and outputs an HR-HSI. Extensive experiments demonstrate the effectiveness of DTUML, showing superior performance and strong competitiveness compared to state-of-the-art methods. Code: https://github.com/yuanchaosu/TGRS-DTUML.
15 January 2025 in IEEE Geoscience and Remote Sensing Letters
Hyperspectral images (HSIs) typically have finer spectral resolution but coarser spatial resolution than multi-spectral images (MSIs). To obtain HSIs with enhanced spatial resolution, considerable emphasis has been placed on achieving hyperspectral super-resolution by fusing HSIs with MSIs in the same scene. However, most existing HSI-MSI fusion methods either rely on prior knowledge of degradation models or require sufficient training data, hindering their practicality and interpretability. This letter proposes a deep multi-stream network (DMSN) for hyperspectral image super-resolution. Specifically, we introduce the Spa-DNet and the Spe-UNet modules to encode spatial and spectral transformations across resolutions. Furthermore, we design the Int-Net to achieve spatial and spectral information interaction, enhancing the model’s performance. Finally, the proposed approach enables high spatial and spectral resolution hyperspectral images. Using the newly designed three-stage training strategy, the network parameters can exhibit the clear physical significance of the degradation process, thereby helping to ensure faithful reconstruction of the desired HSIs. Experimental results with real datasets demonstrate that the proposed DMSN performs better than other methods. The codes will be available at https://github.com/yuanchaosu/dmsn-GRSL.