Multi-focus image fusion technology is to extract different focused regions of the same scene among partially focused images and merge them together to generate a composite image where all objects are clear. Two crucial points to multi-focus image fusion are the effective focus measurement method to evaluate the sharpness of the source images and the accurate segmentation method to extract the focused regions. In conventional multi-focus image fusion methods, the decision map obtained according to the focus measurement is sensitive to mis-registration, or produces an uneven boundary lines. In this paper, the maximum value in the top-hat transform and the bottom-hat transform is used as the gradient measurement value, and the complementary features between multiple scales are used to achieve accurate focus measurement for initial segmentation. In order to obtain a better fusion decision map, a robust image matting algorithm is used to refine the trimap generated by the initial segmentation. Then, make full use of the strong correlation between the source images to optimize the edge regions of the decision map to improve the image fusion quality. Finally, a fusion image is constructed based on the fusion decision map and the source images. We perform qualitative and quantitative experiments on publicly available databases to verify the effectiveness of the method. The results show that compared with several state-of-the-art algorithms, the proposed fusion method can obtain accurate decision maps and achieve better performance in visual perception and quantitative analysis.

This study proposes an innovative network to fuse infrared and visible images, called HitFusion, which uses the cross-feature transformer module and is compatible with high-level vision tasks. Firstly, existing image fusion approaches primarily concentrate on optimizing human visual perception and image metrics. To enhance the performance of the fusion network in subsequent high-level vision tasks, a segmentation network and a corresponding loss are introduced into the fusion network training process. Specifically, we devise a three-stage training strategy to render the fusion network more suitable for high-level vision tasks, guided by the segmentation network and broadening the fusion network's training set to boost its generalization capability. Secondly, current transformer-based image fusion methods neglect the interaction between visible texture features and infrared contrast features. To tackle this, the cross-feature transformer module is proposed, allowing the fusion network to learn the cross-feature correlation and long-range dependencies between source images, thus achieving fusion results with good complementarity. Finally, a dual-branch fusion network is proposed, based on the distinct characteristics of different images, that targets the extraction of deep features from source images utilizing contrast residual and texture enhancement modules to achieve improved fusion results. Extensive experimental results reveal that our HitFusion method excels in both qualitative and quantitative assessments, while also demonstrating superior performance in addressing high-level vision tasks.

Contemporary image fusion methods face challenges in meeting the demands of dim nighttime environments, often accompanied by the concealment of image details in dark regions. In this article, we introduce a novel approach, named LENFusion, which achieves a beneficial interaction between low-light enhancement and image fusion in the form of a feedback loop. LENFusion is primarily divided into three components: luminance adjustment network (LAN), re-enhancement and fusion network (RFN), and luminance feedback network (LFN). The enhancement is performed in two stages. In the initial stage, LAN applies adaptive luminance adjustment to the original visible image. Subsequently, RFN achieves secondary enhancement and feature fusion with a clever combination of dual-attention mechanism, which motivates the fusion results to have high contrast and sharpness. Finally, LFN uses the luminance feedback loss to guide the luminance information of the fused images back to the LAN, effectively avoiding inappropriate enhancement of the images that do not meet the fusion requirements. In addition, we propose a reference-free color loss method for nighttime image fusion. Extensive comparison and generalization experiments have verified the superior fusion performance of LANFusion. Our code is publicly available at: https://github.com/Liling-yang/LENFsuion .

Semantic segmentation of remote sensing images plays a critical role in areas such as urban change detection, environmental protection, and geohazard identification. Convolutional Neural Networks (CNNs) have been excessively employed for semantic segmentation over the past few years; however, a limitation of the CNN is that there exists a challenge in extracting the global context of remote sensing images, which is vital for semantic segmentation, due to the locality of the convolution operation. It is informed that the recently developed Transformer is equipped with powerful global modeling capabilities. A network called TCNet is proposed in this article, and a parallel-in-branch architecture of the Transformer and the CNN is adopted in the TCNet. As such, the TCNet takes advantage of both Transformer and CNN, and both global context and low-level spatial details could be captured in a much shallower manner. In addition, a novel fusion technique called Interactive Self-attention is advanced to fuse the multilevel features extracted from both branches. To bridge the semantic gap between regions, a skip connection module called Windowed Self-attention Gating is further developed and added to the progressive upsampling network. Experiments on three public datasets (i.e., Bijie Landslide Dataset, WHU Building Dataset, and Massachusetts Buildings Dataset) depict that TCNet yields superior performance over state-of-the-art models. The IoU values obtained by TCNet for these three datasets are 75.34% (ranked first among 10 models compared), 91.16% (ranked first among 13 models compared), and 76.21% (ranked first among 13 models compared), respectively.

A single-modal infrared or visible image offers limited representation in scenes with lighting degradation or extreme weather. We propose a multi-modal fusion framework, named SDSFusion, for all-day and all-weather infrared and visible image fusion. SDSFusion exploits the commonality in image processing to achieve enhancement, fusion, and semantic task interaction in a unified framework guided by semantic awareness and multi-scale features and losses. To address the disparity between infrared and visible images in degraded scenes, we differentiate modal features in a unified fusion model. Unlike existing joint fusion methods, we propose an adversarial generative network that refines the reconstruction of low-light images by embedding fused features. It provides feature-level brightness supplementation and image reconstruction to refine brightness and contrast. Extensive experiments in degraded scenes confirm that our approach is superior to state-of-the-art approaches in visual quality and performance, demonstrating the effectiveness of interaction improvement. The code will be posted at: https://github.com/Liling-yang/SDSFusion.