In this paper, a novel fusion network for infrared and visible images is proposed, named Image Pair Learning Fusion Network (IPLF). At present, most of the released deep learning-based fusion models are trained using unsupervised learning. This learning method lacks ground truth and cannot guide network model learning in a targeted manner. First, we propose the use of supervised learning to guide the training of fusion models (IPLF). The generated image pairs are used as ground truth. Second, we propose a model learning strategy using paired images. This learning strategy enhances the complementary constraints of the network model so that the final fused image possesses the rich features of both infrared and visible images. Third, we designed the structure measurement loss function and the edge preservation loss function to ensure that the generated fusion image has rich edge information and comfortable visual effects. In addition, we have introduced a spatial attention module, which can make the final fusion image highlight the target information. Finally, the size of the convolution kernel we use in the first convolution block is $7\ \times\ 7$ , the main purpose is to increase the receptive field. Experiments on the TNO and CVC-14 datasets prove that our proposed method is superior to the existing state-of-the-art methods in terms of qualitative and quantitative evaluation. The source code can be found in https://github.com/depeng6/IPLF .

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network’s ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Image fusion operation is beneficial to many applications and is also one of the most common and critical computer vision challenges. The perfect infrared and visible image fusion results should include the important infrared targets while preserving visible textural detail information as much as possible. A novel infrared and visible image fusion framework is proposed for this purpose. In this paper, the proposed fusion network (MIFFuse) is an end-to-end, multi-level-based fusion network for infrared and visible images. The presented approach makes effective use of the intermediate convolution layer’s output features to preserve the primary image fusion information. We also build a cat_block to swap information between two paths to gain more sufficient information during the convolution steps. To reduce the model’s running time even further, the proposed method that reduces the number of feature channels while maintaining the accuracy of the fusion performance. Extensive experiments on the TNO and CVC-14 image fusion datasets show that our MIFFuse outperforms the other methods in terms of both subjective visual effects and quantitative metrics. Furthermore, MIFFuse is approximately twice as fast as the most recent state-of-the-art methods. Our code and models can be found at https://github.com/depeng6/MIFFuse .

The imbalance between positive and negative samples and the loss of small targets in complex backgrounds are catastrophic for infrared small target detection. To address these issues, we proposed an infrared small target detection method based on weak feature enhancement and target adaptive proliferation (IRSTD-WFETAP). First, we utilized a sparse sampling mechanism and hybrid filtering method (CSHF) to flexibly capture the complex shapes and edge information of small targets while reducing the loss and shift of scarce features. Then, we introduced a multi-scale feature enhancement module (MSFE) that used vertical-horizontal bidirectional attention and multi-scale feature encoding to establish stable feature interaction channels between the encoder and decoder, further enhancing key features of small targets. Additionally, we introduced a data adaptive proliferation strategy (DSAS) to address the imbalance of positive and negative samples, enhancing the generalization and expression capability of the detection datasets. Finally, we proposed a target-background joint loss to alleviate the imbalance issue and help the network converge smoothly. Extensive experiments on NUAA-SIRST, IRSTD-1k and our custom-made dataset demonstrated the effectiveness of the proposed IRSTD-WFETAP method, achieving superior performance in nIoU, Pd, Fa, and F1-Measure compared to the latest methods.

The deep convolutional neural network has achieved great success in the Single Image Super-resolution task. It is obviously that among the well-known super-resolution methods, the deep learning-based algorithms show the most advanced performance. However, the most advanced algorithms currently use complex networks with a large number of parameters, which makes it difficult to apply deep learning algorithms on mobile devices. To solve this problem, we propose a lightweight dual-residual network(LDRN) for single image super-resolution, which has better reconstruction quality than most current advanced lightweight algorithms. Due to its fewer parameters and computational expense, real-time and mobile applications of our networks can be easily realized. On the basis of the residual module, we propose a new residual unit, which uses two depthwise separable (DW) convolution to obtain better balance between feature extraction capacity and lightweight performance. We further design a dual-stream residual block, which contains a multiplication branch and an addition branch. The dual-stream residual block can improve the reconstruction performance more effectively than expanding the network width. In addition, we also designed a new up-sampling module to simplify the previous up-sampling methods. Extensive experimental results show that our network has better reconstruction performance and lightweight performance than most existing state-of-the-art algorithms. Our code is available at https://github.com/Jiangyichun-cust/pytorch-LDRN .