6 May 2025 in IEEE Transactions on Geoscience and Remote Sensing
Hyperspectral image (HSI) encompasses abundant spatial and spectral details, while Light Detection and Ranging (LiDAR) delivers precise elevation data. The amalgamation of HSI and LiDAR data significantly improves the precision of image classification. However, most methods focus solely on spatial features while neglecting frequency domain information, limiting the ability of deep models to characterize land cover. Furthermore, how to establish a sufficient interaction between different modalities is also an important issue. In this paper, we propose a novel multiscale cross-domain fusion network (MCFNet) for joint classification of HSI and LiDAR data. The main idea is that the wavelet transform can provide details at different resolutions simultaneously, supplementing spatial domain information and enriching feature representation. In addition, the multimodal fusion module (MFM) guided by HSI and the cross-domain fusion module (CDFM) strategy are developed to integrate features from diverse modalities and domains, respectively. Specifically, frequency domain features are extracted by discrete wavelet transform, and spatial domain features of the image are captured through a set of convolution operations. Then interactive fusion is performed by MFM and CDFM, and finally the integrated features are categorized using a classification module. Extensive experiments on three widely-used HSI and LiDAR datasets indicate that MCFNet outperforms the SOTA methods. The code will be available at https://github.com/MSFLabX/MCFNet.
1 October 2025 in IEEE Transactions on Neural Networks and Learning Systems
Multimodal recommendation systems integrate diverse multimodal information into the feature representations of both items and users, thereby enabling a more comprehensive modeling of user preferences. However, existing methods are hindered by data sparsity and the inherent noise within multimodal data, which impedes the accurate capture of users' interest preferences. Additionally, discrepancies in the semantic representations of items across different modalities can adversely impact the prediction accuracy of recommendation models. To address these challenges, we introduce a novel diffusion-based contrastive learning (DiffCL) framework for multimodal recommendation. DiffCL employs a diffusion model (DM) to generate contrastive views that effectively mitigate the impact of noise during the contrastive learning phase. Furthermore, it improves semantic consistency across modalities by aligning distinct visual and textual semantic information through stable ID embeddings. Finally, the introduction of the item-item graph (I-I graph) enhances multimodal feature representations, thereby alleviating the adverse effects of data sparsity on the overall system performance. We conduct extensive experiments on three public datasets, and the results demonstrate the superiority and effectiveness of the DiffCL.
13 February 2025 in IEEE Transactions on Neural Networks and Learning Systems
The time-varying Lyapunov equation (TVLE) plays a crucial role in control design and system stability. However, there has been limited research conducted on the time-varying generalized Lyapunov equation in the quaternion field. To tackle the time-varying quaternion generalized Lyapunov equation, a nonlinear noise-resistant zeroing neural network (NNR-ZNN) model with a novel power activation function (NPAF) is devised. The issue of non-commutativity within quaternion is circumvented by utilizing the real representation. The theoretical analyses provide a sufficient explanation for the global stability, fixed-time convergence, and robustness of the NNR-ZNN model. Under several different kinds of noises, the exceptional robustness of the NNR-ZNN model is highlighted by comparison with other existing models. In the end, the successful applications of the NNR-ZNN model to color image fusion and color image denoising confirm the practical value of the NNR-ZNN model.
11 August 2025 in IEEE Journal of Selected Topics in Signal Processing
Due to the complementary characteristics of hyperspectral image (HSI) and multispectral image (MSI), fusing them can generate a high-resolution hyperspectral image (HR-HSI). However, existing convolutional neural network (CNN)- and Transformer-based methods still suffer from challenges such as spectral distortion, insufficient texture representation, and limited ability to integrate global-local information. To address these issues, we propose a spatial invertible network (SINet) with Mamba-convolution. First, we design the multiscale invertible spatial (MISpa) module to effectively model the spatial information of the source image. The MISpa module utilizes Haar wavelet decomposition to separate high and low-frequency spatial components. By integrating invertible neural network (INN), it enhances the extraction of discriminative spatial features while maintaining structural integrity. To mitigate the risk of excessive reliance on spatial information, which may lead to spectral distortion, we employ spectral compression (via average pooling and 3 × 3 convolution) to extract spectral information at each scale of the source image. Through this way, SINet establishes an effective trade-off between detail enhancement and spectral fidelity. Second, we propose a Mamba convolutional fusion module (MCFM) that integrates feature extraction capabilities of Mamba and CNN. This module is designed to adaptively enhance the complementarity between global and local features, thereby guiding the process of feature enhancement. Experiments on both simulated and real-world datasets from quantitative and qualitative perspectives to validate the superiority of the proposed method compared to existing approaches.
6 March 2026 in IEEE Transactions on Geoscience and Remote Sensing
The fusion of hyperspectral image (HSI) and multispectral image (MSI) aims to fully exploit the complementary advantages of both modalities to generate high-resolution HSI. However, existing HSI fusion methods fail to fully utilize frequency domain information when capturing deep correlations between spectral bands, leading to difficulties in accurately preserving spatial texture details. Furthermore, these methods often overlook the intrinsic differential characteristics in HSI fusion, which easily causes spatial–spectral decoupling distortions. To address the aforementioned challenges, this paper proposes a Spatial–Spectral (S2-) Differential Feature Awareness Network (SSDAN) that employs a multi-scale input strategy to fully preserve the rich spatial structures in the source images. Specifically, a Spatial Frequency Fusion Network (SFFN) with a dual-branch architecture is designed to separately model multi-level spatial-frequency features of HSI and MSI, thereby effectively enhancing the joint representation of high-frequency details and global-local structures. Additionally, we introduce a Differential Spatial–Spectral Complementarity Module (DSSCM) that explicitly models modality-specific detailed features and adaptively aligns the dual-branch fused features with the shallow features, effectively mitigating decoupling distortions caused by feature mismatches. The code will be available at https://github.com/MSFLabX/SSDAN.
11 September 2025 in ACM Transactions on Multimedia Computing, Communications, and Applications
Multimodal recommendation systems improve the accuracy of recommendations by integrating information from different modalities to obtain potential representations of users and items. However, existing multimodal recommendation methods often use single user embedding to model users’ interests in different modalities, neglecting multimodal information. Furthermore, the semantics expressed by the same items in different modalities may be inconsistent, leading to suboptimal recommendation performance. To alleviate the impact of these issues, we propose a new multimodal recommendation framework called Learnable Prompts and ID-guided Contrastive Learning (LPIC). Specifically, we introduce a continuously learnable prompt embedding method, incorporating multimodal features of items to model users’ interests in specific modalities. Then, we propose an ID-guided contrastive learning component to enhance historical interaction features in textual, visual, and fused modalities, while aligning text, image, and fused modality to enhance semantic consistency between modalities. Finally, we conduct extensive experiments on three publicly available Amazon datasets to demonstrate the effectiveness of the LPIC framework.