Infrared images can provide prominent targets based on the radiation difference, making them suitable for use in all day and night conditions. On the other hand, visible images can offer texture details with high spatial resolution. Infrared and visible image fusion is promising to achieve the best of both. Conventional frequency or spatial multi-scale transformation methods are good at preserving image details. Deep learning-based methods become more and more popular in image fusion because they can preserve high-level semantic features. To tackle the challenge in extracting and fusing cross-modality and cross-domain information, we propose a Spatial-Frequency Collaborative Fusion (SFCFusion) framework that effectively fuses spatial and frequency information in the feature space. In the frequency domain, source images are decomposed into base and detail layers with existing frequency decomposition methods. In the spatial domain, a kernel-based saliency generation module is designed to preserve spatial region-level structural information. A deep learning-based encoder is employed to extract features from the source images, decomposed images and saliency maps. In the shared feature space, we achieve cross-modality spatial-frequency collaborative fusion through our proposed adaptive fusion scheme. We have conducted experiments to compare our SFCFusion with both conventional and deep learning approaches on TNO, LLVIP and M ³ FD datasets. The qualitative and quantitative evaluation results demonstrate the effectiveness of our SFCFusion. We have further demonstrated the superiority of our SFCFusion in the downstream detection task. Our code will be available at https://github.com/ChenHanrui430/SFCFusion.

Infrared and visible image fusion (IVIF) aims to render fused images that maintain the merits of both modalities. To tackle the challenge in fusing cross-modality information and avoiding texture loss in IVIF, we propose a novel edge-consistent and correlation-driven fusion framework (ECFuse). This framework leverages our proposed edge-consistency fusion module to maintain rich and coherent edges and textures, simultaneously introducing a correlation-driven deep learning network to fuse the cross-modality global features and modality-specific local features. Firstly, the framework employs a multi-scale transformation (MST) to decompose the source images into base and detail layers. Then, the edge-consistent fusion module fuses detail layers while maintaining the coherence of edges through consistency verification. A correlation-driven fusion network is proposed to fuse the base layers containing both modalities’ main features in the transformation domain. Finally, the final fused spatial image is reconstructed by inverse MST. We conducted experiments to compare our ECFuse with both conventional and deep leaning approaches on TNO, LLVIP and ${\mathrm{M}}^3\mathrm{FD}$ datasets. The qualitative and quantitative evaluation results demonstrate the effectiveness of our framework. We also show that ECFuse can boost the performance in downstream infrared–visible object detection in a unified benchmark.