In this article, we propose a novel decomposition method by applying disentangled representation for visible and infrared image fusion (DRF). According to the imaging principle, we perform the decomposition depending on the source of information in the visible and infrared images. More concretely, we disentangle the images into the scene- and sensor modality (attribute)-related representations through the corresponding encoders, respectively. In this way, the unique information defined by the attribute-related representation is closer to the information captured by each type of sensor individually. Thus, the problem of inappropriate extraction of unique information can be alleviated. Then, different strategies are applied for the fusion of these different types of representations. Finally, the fused representations are fed into the pretrained generator to generate the fusion result. The qualitative and quantitative experiments on the publicly available TNO and RoadScene data sets demonstrate the comparable performance of our DRF over the state of the art in terms of both visual effect and quantitative metrics.

Hyperspectral image super-resolution, which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation, aims to improve the spatial resolution of the hyperspectral image, which is beneficial for subsequent applications. The development of deep learning has promoted significant progress in hyperspectral image super-resolution, and the powerful expression capabilities of deep neural networks make the predicted results more reliable. Recently, several latest deep learning technologies have made the hyperspectral image super-resolution method explode. However, a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent. To this end, in this survey, we first introduce the concept of hyper-spectral image super-resolution and classify the methods from the perspectives with or without auxiliary information. Then, we review the learning-based methods in three categories, including single hyperspectral image super-resolution, panchromatic-based hyperspectral image super-resolution, and multispectral-based hyperspectral image super-resolution. Subsequently, we summarize the commonly used hyperspectral dataset, and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively. Moreover, we briefly introduce several typical applications of hyperspectral image super-resolution, including ground object classification, urban change detection, and ecosystem monitoring. Finally, we provide the conclusion and challenges in existing learning-based methods, looking forward to potential future research directions.

Hyperspectral images with tens to hundreds of spectral bands usually suffer from low spatial resolution due to the limitation of the amount of incident energy. Without auxiliary images, the single hyperspectral image super-resolution (SR) method is still a challenging problem because of the high-dimensionality characteristic and special spectral patterns of hyperspectral images. Failing to thoroughly explore the coherence among hyperspectral bands and preserve the spatial–spectral structure of the scene, the performance of existing methods is still limited. In this article, we propose a novel single hyperspectral image SR method termed RFSR, which models the spectrum correlations from a sequence perspective. Specifically, we introduce a recurrent feedback network to fully exploit the complementary and consecutive information among the spectra of the hyperspectral data. With the group strategy, each grouping band is first super-resolved by exploring the consecutive information among groups via feedback embedding. For better preservation of the spatial–spectral structure among hyperspectral data, a regularization network is subsequently appended to enforce spatial–spectral correlations over the intermediate estimation. Experimental results on both natural and remote sensing hyperspectral images demonstrate the advantage of our approach over the state-of-the-art methods.

In recent years, many single hyperspectral image super-resolution methods have emerged to enhance the spatial resolution of hyperspectral images without hardware modification. However, existing methods typically face two significant challenges. First, they struggle to handle the high-dimensional nature of hyperspectral data, which often results in high computational complexity and inefficient information utilization. Second, they have not fully leveraged the abundant spectral information in hyperspectral images. To address these challenges, we propose a novel hyperspectral super-resolution network named SNLSR, which transfers the super-resolution problem into the abundance domain. Our SNLSR leverages a spatial preserve decomposition network to estimate the abundance representations of the input hyperspectral image. Notably, the network acknowledges and utilizes the commonly overlooked spatial correlations of hyperspectral images, leading to better reconstruction performance. Then, the estimated low-resolution abundance is super-resolved through a spatial spectral attention network, where the informative features from both spatial and spectral domains are fully exploited. Considering that the hyperspectral image is highly spectrally correlated, we customize a spectral-wise non-local attention module to mine similar pixels along spectral dimension for high-frequency detail recovery. Extensive experiments demonstrate the superiority of our method over other state-of-the-art methods both visually and metrically.

Although natural image super-resolution methods have achieved impressive performance, single hyperspectral image super-resolution still remains a challenge due to the high dimensionality. In recent years, many single hyperspectral image super-resolution methods adopted the group-convolution strategy to design the network for reducing the computational burden. However, these methods still process all spectral bands at once during the deep feature extraction and reconstruction, which increases the difficulty of fully exploring the inherent data characteristic of hyperspectral images. Moreover, the advanced group-based methods make insufficient exploitation of complementary information contained in different bands, resulting in limited reconstruction performance. In this paper, we propose a novel group-based single hyperspectral image super-resolution method termed GELIN to reconstruct high-resolution images in a group-by-group manner, which alleviates the difficulty of feature extraction and reconstruction for hyperspectral images. Specifically, a spatial-spectral embedding learning module is designed to extract rewarding spatial details and explore the correlations among spectra simultaneously. Considering the high similarity among different bands, a neighboring group integration module is proposed to fully exploit the complementary information contained in neighboring image groups to recover missing details in the target image group. Experimental results on both natural and remote sensing hyperspectral datasets demonstrate that the proposed method is superior to other state-of-the-art methods both visually and metrically.

Deep convolutional neural networks (CNNs) have recently made a considerable achievement in the single-image super-resolution (SISR) problem. Most CNN architectures for SISR incorporate long or short connections to integrate features, and treat them equally. However, they neglect the discrimination of features, and consequently, achieving relatively poor performance. To address this problem, we propose a dense discriminative network that is composed of several aggregation modules (AM). Specifically, the AM merges extraction and integration nodes in a tree structure, which can aggregate features progressively in an efficient way. In particular, we compress and rescale the densely connected information in the aggregation node by modelling the interaction between channels, which shares the same insight with the attention mechanism for improving the discriminative ability of network. Extensive experiments conducted on several publicly available datasets have demonstrated the superiority of our model over state-of-the-art in objective metrics and visual impressions.