26 September 2024 in Expert Systems with Applications
16 February 2026 in IEEE Transactions on Multimedia
CNN-Transformer hybrid architecture have demonstrated remarkable success in image fusion. However, existing studies often oversimplify the respective strengths of CNNs and Transformers, attributing them solely to their abilities to capture local and global information. This characterization is neither precise nor reflective of their fundamental differences, leading to network designs that rely heavily on empirical practices rather than theoretical insights. To address this limitation and enhance the interpretability of fusion networks, we conduct a Fourier analysis to elucidate the distinct roles of CNNs and Transformers from a novel frequency perspective. Our analysis reveals that CNNs and Transformers exhibit contrasting properties in the frequency domain. However, prior CNN-Transformer hybrid architectures have failed to leverage these characteristics, resulting in a loss of low frequency contextual information. Based on this finding, we propose the Frequency Complementary learning and Rebalancing Fusion Network (FCRNet), which leverages the complementary behaviors of CNNs and Transformers to optimize image fusion. Firstly, a frequency complementary learning mechanism is utilized to integrate the strengths of CNNs and Transformers in representation learning, effectively capturing essential low frequency features and rich high frequency details. Secondly, a frequency rebalancing module is designed to address the imbalance between high and low frequency components in the synthesized feature maps, ensuring their frequency distribution aligns with that of natural images. Extensive experiments on multiple benchmarks validate the effectiveness and superiority of our proposed FCRNet. Additionally, experiments on downstream tasks validates the practicality of FCRNet.
30 July 2026 in SSRN Electronic Journal
Multi-Modality Image Fusion (MMIF) aims to reconstruct an informative latent image from complementary sensor observations and is commonly formulated as an inverse problem within a Bayesian framework. Due to its ill-posed nature, the recovery process critically depends on prior knowledge to constrain the solution space and preserve underlying structural characteristics. However, current methods primarily rely on hand-crafted, task-specific priors, which not only introduce subjective biases but also undermine generalizability and robustness across diverse real-world scenarios. To address this issue, we reformulate MMIF within a bias-free Bayesian framework. Each modality is modeled as an independent degradation of the fused image, transforming conventional predefined priors into cross-modality conditional priors and enabling a fully data-driven fusion process. Building on this foundation, we propose the Symmetric Degradation Complementarity Unrolling Network (SDCUNet), which explicitly unrolls the optimization process into a symmetric and interpretable architecture. Each stage of SDCUNet comprises a Degradation Awareness Module (DAM) to model the modality-specific degradation, and a Complementarity Enhancement Module (CEM) to extract structural guidance from cross-modality conditional priors. Additionally, we introduce a Local Gradient-Guided (LGG) pixel-level loss function to adaptively quantify each modality's contribution, further enhancing the model's robustness against complex environmental degradations. Extensive experiments on five infrared-visible and two medical image datasets demonstrate that SDCUNet consistently achieves superior visual quality and quantitative metrics, exhibiting remarkable robustness even under challenging degradation conditions, while improving the downstream detection performance. The code is available at https://github.com/op-dog/SDCUNet.