Multimodal brain signal analysis has shown great potential in decoding complex cognitive processes, particularly in the challenging task of inner speech recognition. This paper introduces an innovative I nner Speech Recognition via Cross-Perception (ISRCP) approach that significantly enhances accuracy by fusing electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data. Our approach comprises three core components: (1) multigranularity encoders that separately process EEG time series, EEG Markov Transition Fields, and fMRI spatial data; (2) a cross-perception expert structure that learns both modality-specific and shared representations; and (3) an attention-based adaptive fusion strategy that dynamically adjusts the contributions of different modalities based on task relevance. Extensive experiments on the Bimodal Dataset on Inner Speech demonstrate that our model outperforms existing methods across accuracy and F1 score.
Automated EEG classification algorithms for seizures can facilitate the clinical diagnosis of epilepsy, enabling more expedient and precise classification. However, existing EEG signal preprocessing methods oriented towards artifact removal and signal enhancement have demonstrated suboptimal accuracy and robustness. In response to this challenge, we propose an Adaptive Dual-Modality Learning Model (ADML) for epileptic seizure prediction by combining time series imaging with Transformer-based architecture. Our approach effectively captures both temporal dependencies and spatial relationships in EEG signals through a specialized attention mechanism. Evaluated on the CHB-MIT and Bonn datasets, our method achieves 98.7% and 99.2% accuracy, respectively, significantly outperforming existing approaches. The model demonstrates strong generalization capability across datasets while maintaining computational efficiency. Cross-dataset validation confirms the robustness of our approach, with consistent performance above 96% accuracy. These results suggest that our dual-modality approach provides a reliable and practical solution for clinical epileptic seizure prediction.
The cross fusion of rehabilitation medicine and computer graphics is becoming a research hotspot. Due to the problems of low definition and unobvious features of the initial video data of medical images, the initial data is filtered and enhanced by adding image preprocessing, including image rotation and contrast enhancement, in order to improve the performance of the tracking algorithm. For the moving barium meal, the discrete point tracking and improved inter frame difference method are proposed; for the position calibration of tissues and organs, the Kernel Correlation Filter (KCF) and Discriminative Scale Space Tracker (DSST) correlation filtering method and the corresponding multi-target tracking method are proposed, and the experimental results show that the tracking effect is better. The two algorithms modify each other to further improve the accuracy of calibration and tracking barium meal flow and soft tissue organ motion, and optimize the whole swallowing process of moving target tracking model.