2 September 2020 in IEEE Transactions on Multimedia
The fusion of multi-band images including far-infrared image (FIRI), near-infrared image (NIRI) and visible image (VISI) primarily suffers from four challenges. One is the problem of simultaneous fusion for multiple images. Most existing methods are oriented towards the fusion of two objects, which is generally achieved with a sequential fusion method. This means that intermediate fusion results are repeatedly integrated with the unprocessed images until all images have been fused. However, this may amplify the blurring effect and even engender artifacts. Second, consistent training labels for image fusion cannot currently be obtained for some types of images (e.g., medical images and multi-band images), which may lead to the failed application of supervised learning methods. Third, the existing methods often do not directly focus on the potential mapping relationship between the original and resulting images, which usually increases the unpredictability of the fusion results. Fourth, redundant features or singularities are often not eliminated in the general fusion process, and both may interfere with or even obscure significant features in the source images. To address the abovementioned problems, this paper proposes a latent representation learning model that can synchronously integrate multi-band images without samples. Specifically, the model can capture the clean and distinctive features of the originals via latent low-rank and sparse embedding. The extracted intrinsic features are projected onto the target fusion space through an assumed mapping relationship. The final results were obtained through the designed optimization algorithm. In addition, numerous experiments were implemented to prove the rationality and feasibility of the proposed fusion model with subjective evaluation, objective indexes and convergence analysis.
12 January 2023 in IEEE Access
This study proposes a new deep neural network to solve the multi-band image synchronous fusion problem (MBF-Net). Unlike other deep learning-based methods, our network architecture design combines the ideas of model-driven and data-driven methods, so it is more interpretable. First, a new multi-band image synchronous fusion model is proposed. The source image in the data fidelity terms and the prior regularization are implicitly represented by the deep learning network and jointly learned from the training data. The proposed model is then solved using a half quadratic splitting (HQS) algorithm and unfolded into a deep fusion network. In addition, a new saliency loss function is proposed to retain thermal radiation information to enhance the fusion effect. Finally, the experimental results on the TNO dataset demonstrated the effectiveness of the proposed MBF-Net.