This paper proposes an effective infrared and visible image fusion method based on a texture conditional generative adversarial network (TC-GAN). The constructed TC-GAN generates a combined texture map for capturing gradient changes in image fusion. The generator in the TC-GAN is designed as a codec structure for extracting more details, and a squeeze-and-excitation module is applied to this codec structure to increase the weight of significant texture information in the combined texture map. The generator loss function is designed by combing the gradient loss and adversarial loss to retain the texture information of the source images. The discriminator brings the texture of the generated image closer to the visible image. To obtain significant texture information from the source images, a multiple decision map-based fusion strategy is proposed using a combined texture map and an adaptive guided filter. Extensive experiments on the public TNO and RoadScene datasets demonstrate that the proposed method is superior to other state-of-the-art algorithms in terms of a subjective evaluation and quantitative indicators.

Multi-focus image fusion is an important technique that aims to generate a single clean image by fusing multiple input images. In this paper, we propose a novel multi-level features convolutional neural network (MLFCNN) architecture for image fusion. In the MLFCNN model, all features learned from previous layers are passed to the subsequent layer. Inside every path between the previous layer and the subsequent layer, we add a 1x1 convolution module to reduce the redundancy. In our method, the source images first are fed to our pre-trained MLFCNN model to obtain the initial focus map. Then, the initial focus map is performed by morphological opening and closing operations and followed by a Gaussian filter to obtain the final decision map. Finally, the fused all-in-focus image is generated based on a weighted-sum strategy with the decision map. The experimental results demonstrate that the proposed method outperforms some state-of-the-art image fusion algorithms in terms of both qualitative and objective evaluations.

The goal of infrared and visible image fusion is to generate an integrated image which can simultaneously preserve more visible detail information and prominent information from the input images. In order to achieve this goal, a novel infrared and visible fusion method based on visual saliency sparse representation and detail injection model is presented. The proposed fusion method contains four steps. The first step is to decompose the source images into base layers and detail layers through a proposed multi-scale decomposition method, which has the advantages of scale awareness and high edge-preservation efficiency. Then, we design a novel sub-fusion rule called visual saliency sparse representation to get the fused base layer. Third, a detail injection model is developed to fuse the detail layers. In this model, we first employ a “max-absolute” scheme to obtain pre-fused images, which are then used as the model items for participating in the fusion process of the detail layers. Finally, the target merged image is achieved through an inverse multi-scale decomposition. Experimental results demonstrate that the proposed method can not only preserve the details and significant information of source images but also enhance the brightness of the fused image, when compared to other state-of-the-art fusion methods.

Multimodal medical image fusion, emerging as a hot topic, aims to fuse images with complementary multi-source information. In this paper, we propose a novel multimodal medical image fusion method based on structural patch decomposition (SPD) and fuzzy logic technology. First, the SPD method is employed to extract two salient features for fusion discrimination. Next, two novel fusion decision maps called an incomplete fusion map and supplemental fusion map are constructed from salient features. In this step, the supplemental map is constructed by our defined two different fuzzy logic systems. The supplemental and incomplete maps are then combined to construct an initial fusion map. The final fusion map is obtained by processing the initial fusion map with a Gaussian filter. Finally, a weighted average approach is adopted to create the final fused image. Additionally, an effective color medical image fusion scheme that can effectively prevent color distortion and obtain superior diagnostic effects is also proposed to enhance fused images. Experimental results clearly demonstrate that the proposed method outperforms state-of-the-art methods in terms of subjective visual and quantitative evaluations.

Pansharpening is used to fuse a panchromatic (PAN) image with a multispectral (MS) image to obtain a high-spatial-resolution multispectral (HRMS) image. Traditional pansharpening methods face difficulties in obtaining accurate details and have low computational efficiency. In this study, a unified pansharpening model based on the band-adaptive gradient and detail correction is proposed. First, a spectral fidelity constraint is designed by keeping each band of the HRMS image consistent with that of the MS image. Then, a band-adaptive gradient correction model is constructed by exploring the gradient relationship between a PAN image and each band of the MS image, so as to adaptively obtain an accurate spatial structure for the estimated HRMS image. To refine the spatial details, a detail correction constraint is defined based on the parameter transfer by designing a reduced-scale parameter acquisition model. Finally, a unified model is constructed based on the gradient and detail corrections, which is then solved by an alternating direction multiplier method. Both reduced-scale and full-scale experiments are conducted on several datasets. Compared with state-of-the-art pansharpening methods, the proposed method can achieve the best results in terms of fusion quality and has high efficiency. Specifically, our method improves the SAM and ERGAS metrics by 17.6% and 21.2% respectively compared to the traditional approach with the best average values, and improves these two metrics by 4.3% and 10.3% respectively compared to the learning-based approach with the best average values.

In this paper, by considering the main objective of multi-focus image fusion and the physical meaning of wavelet coefficients, a discrete wavelet transform (DWT) based fusion technique with a novel coefficients selection algorithm is presented. After the source images are decomposed by DWT, two different window-based fusion rules are separately employed to combine the low frequency and high frequency coefficients. In the method, the coefficients in the low frequency domain with maximum sharpness focus measure are selected as coefficients of the fused image, and a maximum neighboring energy based fusion scheme is proposed to select high frequency sub-bands coefficients. In order to guarantee the homogeneity of the resultant fused image, a consistency verification procedure is applied to the combined coefficients. The performance assessment of the proposed method was conducted in both synthetic and real multi-focus images. Experimental results demonstrate that the proposed method can achieve better visual quality and objective evaluation indexes than several existing fusion methods, thus being an effective multi-focus image fusion method.