The goal of infrared and visible image fusion is to generate an integrated image which can simultaneously preserve more visible detail information and prominent information from the input images. In order to achieve this goal, a novel infrared and visible fusion method based on visual saliency sparse representation and detail injection model is presented. The proposed fusion method contains four steps. The first step is to decompose the source images into base layers and detail layers through a proposed multi-scale decomposition method, which has the advantages of scale awareness and high edge-preservation efficiency. Then, we design a novel sub-fusion rule called visual saliency sparse representation to get the fused base layer. Third, a detail injection model is developed to fuse the detail layers. In this model, we first employ a “max-absolute” scheme to obtain pre-fused images, which are then used as the model items for participating in the fusion process of the detail layers. Finally, the target merged image is achieved through an inverse multi-scale decomposition. Experimental results demonstrate that the proposed method can not only preserve the details and significant information of source images but also enhance the brightness of the fused image, when compared to other state-of-the-art fusion methods.

Infrared small target detection (IRSTD) involves identifying targets that are typically small in spatial extent, have low signal-to-clutter ratios, and are often embedded in dynamic and complex backgrounds, making the task particularly challenging. Benefiting from the powerful feature extraction and multiscale feature fusion characteristics, U-Net performs well in the IRSTD task. However, existing U-Net methods often focus solely on optimizing backbone feature extraction or skip connections, which limits their performance in complex scenes and makes it difficult to recognize small targets effectively. To address this limitation, we propose a novel hierarchical attention fusion network based on the U-Net architecture, namely HAFNet. Specifically, a dual-branch semantic perception module (DSPM) is designed as the feature extraction backbone to enhance contextual semantic interactions. This module integrates dual-branch feature extraction using standard and dilated convolutions while utilizing spatial and channel attention modules (CAMs) to effectively separate small targets from background noise. In addition, we extend the skip connection by merging a hierarchical feature fusion encoder (HFFE) and a hierarchical feature fusion decoder (HFFD). These modules utilize hierarchical attention-guided and encoded feature injection skip connections (ESCs) to achieve effective fusion of multiscale and multilevel semantic features between the encoder and decoder. Extensive experiments on three public datasets (NUAA-SIRST, IRSTD-1K, and NUDT-SIRST) demonstrate that the proposed HAFNet outperforms the existing IRSTD methods and achieves state-of-the-art (SOTA) detection performance. The code will be released on https://github.com/Wangtao-Bao/HAFNet

Multi-sensor infrared and visible image fusion is to extract image features captured by multi-sensor devices to generate a fusion image that contains both infrared thermal radiation and visible texture details. Based on this purpose, this paper proposes a novel infrared and visible image fusion method via double joint edge preservation filter and non-globally saliency gradient operator. First, the source images are finely decomposed by the proposed filter, which has the merits of multi-level scale awareness and double joint edge preservation, into the base, infrared detail, and visible detail layers. Furthermore, to ensure that the overall appearance of the fused image is maximally similar to the source images, a maximum-absolute rule is introduced to process the base layers. Then, a non-globally saliency gradient operator constructed by three different kinds of basic operators is presented to dominate the fusion of the detail layers, aiming to filter out the redundant information and recover the saliency features. Finally, the merged image is obtained by summating the fused base and detail layers. The experimental results demonstrate that the proposed method not only fully extracts the visible details and infrared thermal target of the source images but also generates a fused image with excellent subjective performance and objective indicators compared to other state-of-the-art fusion methods.

Infrared small target detection (IRSTD) plays a vital role in various applications such as surveillance and rescue missions. However, existing methods struggle to effectively integrate contextual information for small targets, with background noise and limited long-distance dependency learning further constraining detection performance. To address these issues, this paper proposes a novel adaptive partial transformer network based on asymmetric U-Net architecture, namely APTNet, which enhances contextual information integration and accurately detects small objects in complex scenes, improving detection performance. Specifically, a Double Residual Attention Block (DRAB) is designed to enhance the contrast of dim targets within complex backgrounds through channel and spatial attention mechanisms, thereby improving discrimination capability for small targets. Furthermore, leveraging the context pooling connections and DRAB output features, an Adaptive Partial Transformer Block is proposed. This block first proposes an adaptive channel splitting factor, achieving contextual information integration and long-distance dependency learning with low computational overhead. Extensive experiments on two public datasets demonstrate that the proposed method significantly outperforms the existing state-of-the-art IRSTD methods in terms of intersection over union (IoU), probability of detection (P_d), and false alarm rate (F_a). The implementation of our code can be accessed at https://github.com/Wangtao-Bao/APTNet.

Infrared small target detection (IRSTD) is a challenging task due to the small size and low contrast of targets in infrared images. Traditional methods rely on extensive prior knowledge and exhibit poor robustness against complex backgrounds. Although CNN-based methods have made remarkable progress in IRSTD, effectively extracting and fully utilizing features at different levels remains challenging. To address this challenge, we propose two strategies for IRSTD: 1) multi-scale perception and 2) cross-attention feature fusion. Based on these strategies, a multi-scale perception and cross-attention feature fusion network is constructed, named MPCNet. Specifically, a multi-scale perception module in the encoder is designed to capture rich contextual information and enhance the localization perception of small targets. For cross-attention feature fusion, a global semantic-aware fusion module in the encoder and a semantic-guided cross-attention fusion module in the decoder is devised, respectively. The former achieves more refined feature fusion by narrowing the semantic gap between features at different levels, while the latter further captures target features accurately by enhancing the semantic associations between features in the encoder and decoder. Experimental results verify the effectiveness of the proposed MPCNet compared to other state-of-the-art (SOTA) IRSTD methods in both quantitative and qualitative evaluations. The code will be released on https://github.com/Wangtao-Bao/MPCNet.

Infrared and visible image fusion aims to obtain an image that can retain the prominent infrared target and the detailed texture information from the source images. Most scale filter-based decomposition methods mainly attempt to extract more detailed features by increasing decomposition layers, and they ignore fully considering the intrinsic properties of the original images and the influence of noises. To solve this issue and obtain better fusion & detection performance, this paper proposes a fusion method via a globalet filter and detail enhancement model to construct a target-aware fusion framework. The globalet filter is first comprehensively considered from three perspectives based on global point-to-point optimization: target brightness, removal of gradients of large-small scales (noise reduction), and preservation of texture details. Mathematically, three limited error measure equations are constructed between the target output and the source image in the shape of the $L_{2}$ -norm and the first-order derivative difference. Next, a weighted average operator and a detail enhancement model are proposed to guide the corresponding sub-layers. This model creates connections between the detail layers and the input images so that the fused detail layer contains the best pixel region from these images to the greatest extent possible after the “maximum absolute” rule. As a result, the fused image is reconstructed by adding the previously obtained sub-images. Extensive experimental results demonstrate that our method outperforms state-of-the-art fusion methods, particularly in highlighting infrared targets, preserving substantial details, and producing average detection accuracy of exceeds 98%.