Multi-focus image fusion (MFIF) is a fundamental task in image processing. It generates an all-in-focus image through multiple partly focused source images. There are common schemes that are based on focused region detection and activity level measurement, and fusion rule. We found that it is difficult to directly map between source images and focus maps. In this study, we investigate the use of U-shape networks for the end-to-end modeling of MFIF. The novelty of our framework is two-fold. First, it uses a U-shaped network as a feature extractor that captures low-frequency information through feature extraction, and high-frequency texture information through high-frequency texture extraction; Compared with the common convolutional neural network, the proposed network has better representation ability, so that the most visually distinctive features can be extracted, fused, and enhanced. Second, the hybrid objective with.1 and perceptual losses enables the framework to yield fused results that are consistent with human beings’ perception. In addition, we employ a weighted strategy to merge the chrominance components in the YCbCr color space so that color distortion is almost eliminated in the color fused result. We investigate the performance through extensive experiments to verify the effectiveness of the proposed method. Through qualitative and quantitative assessment, the proposed method has performance comparable to recent state-of-the art methods.
The objective of pan-sharpening is to effectively fuse high-resolution panchromatic (PAN) images with limited spectral information and low-resolution multispectral (LR-MS) images, thereby generating a fused image with a high spatial resolution and rich spectral information. However, current fusion techniques face significant challenges, including insufficient edge detail, spectral distortion, increased noise, and limited robustness. To address these challenges, we propose a multi-frequency spectral–spatial interaction enhancement network (MFSINet) that comprises the spectral–spatial interactive fusion (SSIF) and multi-frequency feature enhancement (MFFE) subnetworks. The SSIF enhances both spatial and spectral fusion features by optimizing the characteristics of each spectral band through band-aware processing. The MFFE employs a variant of wavelet transform to perform multiresolution analyses on remote sensing scenes, enhancing the spatial resolution, spectral fidelity, and the texture and structural features of the fused images by optimizing directional and spatial properties. Moreover, qualitative analysis and quantitative comparative experiments using the IKONOS and WorldView-2 datasets indicate that this method significantly improves the fidelity and accuracy of the fused images.
28 March 2024 in IEEE Transactions on Computational Imaging
Defocus blur estimation requires high-precision detection between the homogeneous region and transition edge. This paper develops a novel progressive design that effectively addresses this challenge. Our multi-interactive scheme could gradually learn the characteristics of degraded input and divide complex defocus blur estimation into more manageable subnetworks. Specifically, we equally degrade the source inputs and combine them with complementary information subnetworks. In the first two stages, feature interactive modules are introduced to achieve the purpose of information interaction between different features. One challenge in multi-stage networks is transmitting information features between stages, which led to the development of the supervision-guided attention module. Taking into consideration the intricacies associated with neural network design and the pronounced affinity of defocus and focus characteristics with global semantic information, in the final stage, we opt to directly input the original image, after significant affinity-based feature weighting, into the network. This strategic incorporation of global semantic information serves to mitigate the challenges posed by feature concatenation artifacts and noise encountered in the preceding two stages, thereby bolstering the accuracy of the model.
Pan-sharpening aims to generate high-resolution multispectral (HRMS) images by combining high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) data, while maintaining the symmetry of spatial and spectral characteristics. Traditional convolutional neural networks (CNNs) struggle with global dependency modeling due to local receptive fields, and Transformer-based models are computationally expensive. Recent Mamba models offer linear complexity and effective global modeling. However, existing Mamba-based methods lack sensitivity to local feature variations, leading to suboptimal fine-detail preservation. To address this, we propose a Conditional Skipping Mamba Network (CSMN), which enhances global-local feature fusion symmetrically through two modules: (1) the Adaptive Mamba Module (AMM), which improves global perception using adaptive spatial-frequency integration; and (2) the Cross-domain Mamba Module (CDMM), optimizing cross-domain spectral-spatial representation. Experimental results on the IKONOS and WorldView-2 datasets demonstrate that CSMN surpasses existing state-of-the-art methods in achieving superior spectral consistency and preserving spatial details, with performance that is more symmetric in fine-detail preservation.