19 July 2024 in IEEE Transactions on Intelligent Transportation Systems
Infrared and visible image fusion (IVF) aims to generate a fused image with abundant texture details and salient thermal radiation targets, which can not only preserve the necessary scene information for traffic vision tasks, but also highlight the imperceptible targets that are crucial in intelligent transportation system (ITS). However, the existing image fusion methods often lack the information interactions between cross-modal features and among cross-granularity features, and they usually ignore the importance of edge information to the image, which affects the quality of the fused image. To this end, this study proposes an IVF network based on cross-modal multi-granularity information interaction and edge guidance, termed as CEFusion. On the one hand, a triple-branch scene fidelity module is designed to fuse the different modal features extracted by the encoder. This module can adequately mine difference information and infrared salient information of the cross-modal features through cross-modal information interaction. On the other hand, a progressive cross-granularity interaction feature enhancement module is employed to achieve the information interaction among cross-granularity features, which can further enrich the texture and structure information in the fused features. In addition, a novel edge loss function is proposed to guide the network to retain the edge information from source images. Extensive comparative and generalization experiments demonstrate that our CEFusion superior to the state-of-the-art methods in preserving texture details and thermal radiation targets. More importantly, the performance of our method in the high-level vision task suggests that it can provide reliable assistance for the ITS applications.
29 April 2026 in IEEE Transactions on Image Processing
Polarization and intensity images fusion (PIF) has extracted extensive attentions as it can generate images with clear scene information and salient texture details of the object surface that are important for downstream applications. However, existing deep learning-based PIF methods usually lack interpretability and ignore the interactions among multi-modal features. To this end, we propose a novel interpretable low-rank sparse representation guided fusion network for polarization and intensity images (termed LSRNet). Specifically, a low-rank sparse representation deep unfolding module is designed to acquire the base and detail features of the source images, with the ability of improving the interpretability of the network. In addition, a cross-modal connection complementary feature extraction module is proposed, which aims to establish dependency among features of multi-modalities to fully extract complementary features of the source images. In order to demonstrate the validity of our LSRNet and take into account shortcomings of existing datasets for PIF, a multi-scene polarization and intensity image dataset, named MSPI dataset, is constructed, which includes 1034 high-resolution aligned image pairs. According to the best of our knowledge, this is the most comprehensive dataset for PIF that with a large number of image pairs, high resolution and multiple scene types. Extensive experiments on our MSPI dataset and two publicly available datasets (i.e., 12CFC and HCP) demonstrate the superior fusion performance, generalization ability, and desirable running efficiency of our LSRNet. Our codes and dataset will be publicly available at https://github.com/thebinyang/LSRNet.