The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and (3) performance analysis of medical image fusion on mainly data set. Finally, the conclusion of this paper is that the current multimodal medical image fusion research results are more significant and the development trend is on the rise but with many challenges in the research field.

Infrared visible image fusion plays a central role in multi-modal image fusion. By integrating feature information, we obtain more comprehensive and richer visual data to enhance image quality. However, current image fusion methods often rely on intricate networks to extract parameters from multimodal source images, making it challenging to leverage valuable information for high-quality fusion results completely. In this research, we propose a Poolformer-CNN dual-branch feature extraction fusion network for the fusion of infrared and visible images, termed PFCFuse. This network fully exploits key features in the images and adaptively preserves critical features in the images. To begin with, we provide a feature extractor with a dual-branch poolformer-cnn, using poolformer blocks to extract low-frequency global information, where the basic spatial pooling procedures are used as a substitute for the attention module of the Transformer. Secondly, the model is designed with an adaptively adjusted a-Huber loss, which can stably adjust model parameters and reduce the influence of outliers on model predictions, thereby enhancing the model’s robustness while maintaining precision. Compared with state-of-the-art fusion models such as U2Fusion, RFNet, TarDAL, and CDDFuse, we obtain excellent experimental results in both qualitative and quantitative experiments. Compared to the latest dual-branch feature extraction, CDDFuse, our model parameters are reduced by half. Code available at https://github.com/HXY13/PFCFuse-Image-Fusion.

Wanying Xie, Yang Feng, Shuhao Gu, Dong Yu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.