The current advanced hyperspectral super-resolution methods utilize Convolutional Neural Networks (CNNs) that are either deeper or wider. These networks are designed to acquire end-to-end mapping capability, facilitating the transformation from Low-Resolution Hyperspectral Images (LR-HSI) and High-Resolution Multispectral Images (HR-MSI) to High-Resolution Hyperspectral Images (HR-HSI). The existing methods lack the capability to capture details and structures in the image effectively, while multi-input and multi-output methods can address this issue efficiently. Therefore, this paper proposes a novel network architecture named Multi-Input Multi-Output Spatial-Spectral Transformer (MIMO-SST). To apply the multi-input and multi-output methods in HSI fusion, specifically integrating the spatial-spectral information of LR-HSI and HR-MSI, we introduce multi-head feature map attention, multi-head feature channel attention, and a multi-scale convolutional gated feedforward network, constructing the proposed Mixture spatial-spectral Transformer. Moreover, to enhance the expressive power of image edges and recover the sharpened structure details, this study incorporates a novel wavelet-based high-frequency loss into the ultimate comprehensive loss, with the objective of refining the reconstruction of high-frequency details. Experimental studies on three simulated datasets and one real-world dataset demonstrate that the proposed method in this study outperforms contemporary state-of-the-art methods in terms of performance. It is noteworthy that our method exhibits a 0.85 dB improvement in terms of the PSNR metric on the CAVE dataset compared to state-of-the-art methods. Our code is publicly available at https://github.com/Freelancefangjian/MIMO-SST.

The fusion of high-resolution multispectral image (HrMSI) and low-resolution hyperspectral image (LrHSI) has been acknowledged as a promising method for generating a high-resolution hyperspectral image (HrHSI), which is also termed to be an essential part for precise recognition and cataloguing of the underlying materials. In order to improve the fusion of the LrHSI and HrMSI performance, in this article, we propose a novel nonnegative matrix factorization inspired deep unrolling networks (NMF-DuNet) for fusing LrHSI and HrMSI. For this aim, initially, a variational fusion model regularized by nonnegative sparse prior is proposed and then is solved through the gradient descent optimization method and unrolled towards the deep network. The nonnegative coefficient matrices and orthogonal of the proposed transform coefficients constraints are both incorporated into the proposed method. Moreover, the fusion of HrMSI and LrHSI heavily depends on an imaging model that explains the degeneracy of HSI in the spectral and spatial regions. Practically, the imaging model is often unknown. The degradation model is represented implicitly via a proposed network, and both the degradation model and sparse priors are jointly optimized through the training process of the proposed network. Instead of being hand-crafted, all the parameters of NMF-DuNet are learned end-to-end. Compared to the previous state-of-the-art model-based and learning-based fusion approaches, the hardware-friendly proposed NMF-DuNet outperforms both the model-based and learning-based fusion approaches and requires a far smaller number of trainable parameters and storage space while preserving the real-time performance.

Due to the physical boundaries, fusing low spatial resolution hyperspectral (LrHSI) with high spatial resolution multispectral (HrMSI) images is a hot and promising area for obtaining hyperspectral that have high spatial-spectral resolution images (HrHSI). Effectively formulating the fundamental features of hyperspectral images (HSI), such as global spectral correlation, non-local spatial correlation, as well spatial-spectral correlation, is complex in HSI-MSI fusion. Moreover, the fusion process is highly affected by the degradation systems, where these systems are not known in real scenarios. To this end, in this paper, we proposed a Model-Guided Deep Unfolded Fusion Network with Non-local Spatial-Spectral Priors (MGDuNLSS-net) that can maintain the essential features of the HSIs and implicitly estimates the degradation process in an adequate running time. Specifically, the proposed method is designed based on subspace representation in an iterative manner and unrolling its steps toward a deep network as an end-to-end framework. This approach contains two sub-modules, fusion (non-local spatial-spectral block (NLSSB)) and imaging system sub-modules. The former sub-module is proposed to exploit the images' intrinsic characteristics to improve the preservation of spectral and spatial details. NLSSB contains two non-local self-similarity (NLSS) layers embedded between two bidirectional Simple recurrent unit (BSRU) layers. The recurrent calculation, as well as refined components to maintain the global spectral correlation, are the light recurrence operation and highway network, while 3D convolutions in the BSRU can retain the spatial-spectral correlation. The NLSS layer can efficiently and effectively model long-range spatial contexts, which is designed based on criss-cross attention. The later sub-module is used to refine the prediction of the degradation process at any iteration via back-projecting the estimated fused image to the observed pair, which can ensure the good performance of fusion. Compared with state-of-the-art fusion approaches, three remote sensing datasets are used to validate the proposed approach's performance.

Content-based image retrieval (CBIR) is a technique uses visual contents such as color, texture and shape to search images from large scale image databases according to users' interest. In a CBIR, visual image content is represented in form of image features, which are extracted automatically and there is no manual intervention, thus eliminating the dependency on humans in the feature extraction stage. Recent studies in CBIR get the similarity results and retrieve images based on one type of feature which are color, texture or shape. In this study authors proposed a fusion based retrieval model for merging results taken from color and texture image features based different fusion methods. After implementing our proposed retrieval model on Wang image dataset which widely used in CBIR, the results show that CombMEAN fusion approach has the best and high precision value and outperformed both individual color and texture retrieval model in both top10 and top20 retrieved images.

The fusion-based super-resolution of hyperspectral images (HSI) draws more and more attention in order to surpass the hardware constraints intrinsic to hyperspectral imaging systems in terms of spatial resolution. Low-resolution (LR) HSI is combined with a high-resolution multispectral image (HR-MSI) to achieve HR-HSI. In this paper, we propose multi-resolution details enhanced attentive dual-UNet (MDA-DUNet) to improve the spatial resolution of HSI. The entire network contains two branches. The first branch is the wavelet detail extraction module, which performs discrete wavelet transform on MSI to extract spatial detail features and then passes through the encodingdecoding. Its main purpose is to extract the spatial features of MSI at different scales. The latter branch is the spatio-spectral fusion module, which aims to inject the detail features of the wavelet detail extraction network into the HSI to reconstruct the HSI better. Moreover, this network uses an asymmetric feature selective attention model to focus on important features at different scales. Extensive experimental results on both simulated and real data show that the proposed network architecture achieves the best performance compared with several leading HSI super-resolution methods in terms of qualitative and quantitative aspects.

In satellite remote sensing, the complementary nature of hyperspectral (HSI) and multispectral (MSI) imagery necessitates their fusion to enhance both spatial and spectral resolution. However, the inherent misalignment between these datasets, due to differences in acquisition conditions, poses a significant challenge. This study presents a novel approach, the Deep Unfolding Network Enhanced by Transformer Priors (DUNET), to address the simultaneous registration and fusion of HSI and MSI. Unlike conventional deep fusion methods, which are often treated as opaque “black boxes”, DUNET incorporates the deep unfolding method, leveraging mutual information and deep priors to facilitate a better degradation model-informed fusion process. The proposed network incorporates hybrid attention transformers and spatial frequency modules to fully exploit the spatial-spectral information of HSI, resulting in a more accurate and detailed representation of the scene. We conducted extensive quantitative and visual experiments on three standard HSI datasets. The results demonstrate that our proposed DUNET method outperforms existing mainstream algorithms in the field of remote sensing image fusion, showcasing its effectiveness. Specifically, our proposed method achieves improvements of 3.4 dB, 5.1 dB, and 8.2 dB in terms of peak signal-to-noise ratio (PSNR) compared to the latest methods on the ICVL, Chikusei, and Houston datasets, respectively.