The integration of multi-modality images significantly enhances the clarity of critical details for object detection. Valuable semantic data from object detection enriches the fusion process of these images. However, the potential reciprocal relationship that could enhance their mutual performance remains largely unexplored and underutilized, despite some semantic-driven fusion methodologies catering to specific application needs. To address these limitations, this study proposes a mutually reinforcing, dual-task-driven fusion architecture. Specifically, our design integrates a feature-adaptive interlinking module into both image fusion and object detection components, effectively managing the inherent feature discrepancies. The core idea is to channel distinct features from both tasks into a unified feature space after feature transformation. We then design a feature-adaptive selection module to generate features rich in target semantic information and compatible with the fusion network. Finally, effective combination and mutual enhancement of the two tasks are achieved through an alternating training process. A diverse range of swift evaluations is performed across various datasets to corroborate the potential efficiency of our framework, actualizing visible advancements in both fusion effectiveness and detection accuracy.

In the field of image fusion, the fusion of infrared and visible images emerges as a crucial work, retaining the inherent characteristics of each source image for the creation of high-quality fusion images. However, existing fusion algorithms primarily prioritize visual performance and statistical metrics, often neglecting the requisites of high-level vision tasks. To bridge this gap, our paper proposes SIGFusion, an end-to-end method for infrared and visible image fusion. SIGFusion not only considers the visual performance and statistical metrics of the fused image but also focuses on meeting the demands of subsequent high-level vision tasks, thereby enhancing performance in fusion and high-level vision tasks. Concretely, we define significant semantic information as salient targets presenting in both infrared and visible images. By utilizing targets, our fusion network is guided to enhance these salient targets. In contrast to the isolated employ of either infrared or visible images, our target-based method comprehensively utilizes the semantic information within the image. Additionally, the utilization of target-based self-attention loss enhances the extraction of vital information from the source image. It is worth noting that masks are only necessary during the training stage and are automatically generated through detection algorithms. We conduct experiments in both image fusion and high-level vision tasks. Experimental results demonstrate the effectiveness of SIGFusion in both infrared and visible image fusion tasks, as well as in subsequent high-level vision tasks. The source code is available at https://github.com/poison-pig/SIGFusion.

Infrared and visible image fusion is an image enhancement technique that generates a single image with rich textures and significant objectives in a variety of scenarios, providing great convenience for human discrimination and computer recognition. However, in low-light environments, low-intensity visible images tend to blur valuable information, and these details are often ignored during image fusion, resulting in the loss of important information. Although existing methods take into account the damage of low illumination and highlight the illumination in the fusion process, a large amount of structural information is lost in the process of adjusting illumination, resulting in the lack of texture details and poor performance in high-level vision tasks. To address the above challenges, this paper proposes a structure-aware image fusion method for low illumination scenes, called SLFusion, which enhances the illumination while reducing the loss of structural information, leading to a fused image with richer texture details. We first design an illumination enhancement module to separate the degraded illumination from the scene information in the visible image, and mine more details from the low-intensity regions. Based on the fact that image edge information has a good capability of modeling structures, we design an edge extraction network for low-light visible images to model the structural information, which can accurately highlight important structural information and inject it into the fusion image. The proposed method produces fusion results that not only have good visual perception, but also minimize the loss of structural information. Extensive experiments on benchmark datasets demonstrate that the proposed method outperforms state-of-the-art (SOTA) methods in terms of visual quality, quantitative metrics as well as advanced vision tasks.

The fusion classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has gained widespread attention because of its ability to obtain more comprehensive spatial and spectral information. However, the heterogeneous gap between HSI and LiDAR data also adversely affects the classification performance. Despite the excellent performance of traditional multimodal fusion classification models, language information containing much linguistic priori knowledge to enrich visual representations needs to be addressed. Therefore, we design a Spectral-Spatial-Language fusion network (S ² LFNet), which can fuse visual and language features to broaden the semantic space using linguistic priori knowledge commonly shared between spectral features and spatial features. First, we propose a dual-channel cascaded image fusion encoder (DCIFencoder) for visual feature extraction and progressive feature fusion of different levels for HSI and LiDAR data. Then, three aspects of Text data are designed to extract linguistic priori knowledge using the Text encoder. Finally, contrastive learning is utilized to construct a unified semantic space, and Spectral-Spatial-Language fusion features are obtained for classification tasks. We evaluate the classification performance of the proposed S ² LFNet on three datasets through extensive experiments, and the results show that it outperforms the state-of-the-art fusion classification methods.

The purpose of infrared and visible image fusion is to create a single image containing the texture details and significant object information of the source images, particularly in challenging environments. However, existing image fusion algorithms are generally suitable for normal scenes. In the hazy scene, a lot of texture information in the visible image is hidden, the results of existing methods are filled with infrared information, resulting in the lack of texture details and poor visual effect. To address the aforementioned difficulties, we propose a haze-free infrared and visible fusion method, termed HaIVFusion, which can eliminate the influence of haze and obtain richer texture information in the fused image. Specifically, we first design a scene information restoration network (SIRNet) to mine the masked texture Information in visible images. Then, a denoising fusion network (DFNet) is designed to integrate the features extracted from infrared and visible images and remove the influence of residual noise as much as possible. In addition, we use color consistency loss to reduce the color distortion resulting from haze. Further-more, we publish a dataset of hazy scenes for infrared and visible image fusion to promote research in extreme scenes. Extensive experiments show that HaIVFusion produces fused images with increased texture details and higher contrast in hazy scenes, and achieves better quantitative results, when compared to state-of-the-art image fusion methods, even combined with state-of-the-art dehazing methods.