RGBT tracking has received extensive attention as a research hotspot, benefiting from the complementarity of visible (RGB) and thermal (T) modalities, which can better handle tracking tasks in harsh environments. Existing studies typically deploy the same structure to extract features of heterogeneous modalities with different properties, which may be difficult to simultaneously search for mining strategies suitable for each modality, resulting in insufficient feature representation. And, with the success of Transformer, how to jointly consider the heterogeneity and global mining to facilitate better integration of heterogeneous modalities is also a crucial issue worth exploring. To solve these problems, we propose a unique differential reinforcement and global collaboration network for RGBT tracking, which includes a parallel backbone, modal differential reinforcement (MDR), and global mining collaboration module (GMCM). Specifically, MDR introduces raw data to control adaptive differential reinforcement for both modalities via a modality-sharing augmentation unit and a modality-aware gating unit. The data-driven design makes it easier for the two modalities to search for the respective appropriate mining strategies, further effectively enhancing the respective feature representations. GMCM first mines intra-modal and cross-modal global information separately, and then performs an adaptive ensemble operation to balance the two. This operation learns intra-modal and cross-modal global weights in RGB and T modalities to better integrate heterogeneous modalities based on heterogeneity and global cues. Extensive experiments on three datasets demonstrate that the performance of the proposed network is comparable to state-of-the-art RGBT trackers.

A fusion algorithm based on target extraction for infrared image (IIR) and visible image fusion in the nonsubsampled contourlet transform (NSCT) domain is proposed. Commonly, the target information in IIR is important; in order to fully retain the target information in a final fused image, first, use maximum between-class variance method to segment IIR, such that the target regions with salient objects are extracted to produce the background and target images. Next, the visible and background images are decomposed to a series of low-pass and band-pass images by NSCT, respectively. Then, fuse the obtained images to produce the fused background image by different strategies in each band, in which Gaussian fuzzy logic is used to produce the low-pass coefficient; the spatial frequency of each band-pass image is used to determine the linking strength β value of pulse coupled neural network structure, and the result is used to fuse the band-pass images. Eventually, the fused image is produced combining the target image and the fused background image. The experiments show that this algorithm can retain more background details of the two images and highlight the target in the infrared image more effectively, as well as obviously improve the visual effect of the fusion image.

Multi-focus image fusion (MFIF) is a fundamental task in image processing. It generates an all-in-focus image through multiple partly focused source images. There are common schemes that are based on focused region detection and activity level measurement, and fusion rule. We found that it is difficult to directly map between source images and focus maps. In this study, we investigate the use of U-shape networks for the end-to-end modeling of MFIF. The novelty of our framework is two-fold. First, it uses a U-shaped network as a feature extractor that captures low-frequency information through feature extraction, and high-frequency texture information through high-frequency texture extraction; Compared with the common convolutional neural network, the proposed network has better representation ability, so that the most visually distinctive features can be extracted, fused, and enhanced. Second, the hybrid objective with.1 and perceptual losses enables the framework to yield fused results that are consistent with human beings’ perception. In addition, we employ a weighted strategy to merge the chrominance components in the YCbCr color space so that color distortion is almost eliminated in the color fused result. We investigate the performance through extensive experiments to verify the effectiveness of the proposed method. Through qualitative and quantitative assessment, the proposed method has performance comparable to recent state-of-the art methods.

This paper proposed an effective multifocus color image fusion algorithm based on nonsubsampled shearlet transform (NSST) and pulse coupled neural networks (PCNN); the algorithm can be used in different color spaces. In this paper, we take HSV color space as an example, H component is clustered by adaptive simplified PCNN (S-PCNN), and then the H component is fused according to oscillation frequency graph (OFG) of S-PCNN; at the same time, S and V components are decomposed by NSST, and different fusion rules are utilized to fuse the obtained results. Finally, inverse HSV transform is performed to get the RGB color image. The experimental results indicate that the proposed color image fusion algorithm is more efficient than other common color image fusion algorithms.