Image fusion is an important task for computer vision as a diverse range of applications are benefiting from the fusion operation. The existing image fusion methods are largely implemented at the pixel level, which may introduce artifacts and/or inconsistencies while the computational complexity is relatively high. In this paper, we propose a symmetric encoder-decoder with residual block (SEDRFuse) network to fuse infrared and visible images for night vision applications. At the training stage, the SEDRFuse network is trained to create a fixed feature extractor. At the fusing stage, the trained extractor is utilized to extract the intermediate and compensation features, which are generated by the residual block and the first two convolutional layers from the input source images, respectively. Two attention maps, which are derived from the intermediate features, are then multiplied by the intermediate features for fusion. The salient compensation features obtained through element-wise selection are passed to the corresponding deconvolutional layers for processing. Finally, the fused intermediate features and the selected compensation features are decoded to reconstruct the fused image. Experimental results demonstrate that the proposed fusion solution, i.e., SEDRFuse, outperforms the state-of-the-art fusion methods in terms of both subjective and objective evaluations.

Traditional image fusion focuses on selecting an effective decomposition approach to extract representative features from the source image and attempts to find appropriate fusion rules to merge extracted features respectively. However, the existing image decomposition tools are mostly based on kernels or global energy-optimized functions limiting the performance of the wide range of image contents. This paper proposes a novel infrared and visible image fusion method based on deep decomposition network and saliency analysis (named DDNSA). First, the modified residual dense network (MRDN) is trained with a publicly available dataset to learn the decomposition process. Second, the structure and texture features of source images are separated by the trained decomposition network. Then, according to the characteristics of the above features, we construct the combination of local and global saliency maps by using stacked sparse autoencoder and visual saliency mechanism to fuse the structural features. Besides, we propose a bi-direction edge-strength fusion strategy for merging the texture features. Finally, the resultant image is reconstructed by combining the fused structure and texture features. The experimental results confirm that our proposed method outperforms the state-of-the-art methods in both visual perception and objective evaluation.

Pansharpening refers to the use of a panchromatic image to improve the spatial resolution of a multi-spectral image while preserving spectral signatures. However, existing pansharpening methods are still unsatisfactory at balancing the trade-off between spatial enhancement and spectral fidelity. In this paper, a multi-scale and multi-stream fusion network (named MMFN) that leverages the multi-scale information of the source images is proposed. The proposed architecture is simple, yet effective, and can fully extract various spatial/spectral features at different levels. A multi-stage reconstruction loss was adopted to recover the pansharpened images in each multi-stream fusion block, which facilitates and stabilizes the training process. The qualitative and quantitative assessment on three real remote sensing datasets (i.e., QuickBird, Pléiades, and WorldView-2) demonstrates that the proposed approach outperforms state-of-the-art methods.

Spectral and spatial fidelity remains a longstanding challenge in the field of pansharpening, which aims to generate high-resolution multispectral (HRMS) images by integrating high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images. This study proposes a high-fidelity pansharpening network that utilizes bidirectional trigeminal pyramid decoding of features encoded by a CNN-Transformer architecture. Specifically, local and global features at multiple scales are initially extracted using a CNN-Transformer encoder to facilitate multi-scale feature fusion. Subsequently, we design a decoder based on bidirectional trigeminal pyramids to achieve a high-fidelity fusion output. One reverse decoding pyramid decodes the fused features of LRMS and PAN images from the encoder. One spectral feature pyramid is employed to enhance the spectral information of the reverse decoding pyramid, while the last spatial feature pyramid is utilized to enrich the spatial information, thereby improving the overall spectral and spatial fidelity of the fused output. Furthermore, content-guided attention (CGA) is incorporated to adaptively integrate the spectral and spatial feature pyramids into the reverse decoding pyramid. Extensive experiments demonstrate that our network surpasses the comparative state-of-the-art (SOTA) methods in both qualitative and quantitative evaluations. The code is available at https://github.com/songvvvv/pansharpening.