Objective: Electroencephalography (EEG) is among the most widely used and inexpensive neuroimaging techniques. Compared to the CNN or RNN based models, Transformer can better capture the temporal information in EEG signals and focus more on global features of the brain's functional activities. Importantly, according to the multiscale nature of EEG signals, it is crucial to consider the multi-band concept into the design of EEG Transformer architecture. Methods: We propose a novel M ulti-band EE G T ransformer (MEET) to represent and analyze the multiscale temporal time series of human brain EEG signals. MEET mainly includes three parts: 1) transform the EEG signals into multi-band images, and preserve the 3D spatial information between electrodes; 2) design a Band Attention Block to compute the attention maps of the stacked multi-band images and infer the fused feature maps; 3) apply the Temporal Self-Attention and Spatial Self-Attention modules to extract the spatiotemporal features for the characterization and differentiation of multi-frame dynamic brain states. Results: The experimental results show that: 1) MEET outperforms state-of-the-art methods on multiple open EEG datasets (SEED, SEED-IV, WM) for brain states classification; 2) MEET demonstrates that 5-bands fusion is the best integration strategy; and 3) MEET identifies interpretable brain attention regions. Significance: MEET is an interpretable and universal model based on the multiband-multiscale characteristics of EEG. Conclusion: The innovative combination of band attention and temporal/spatial self-attention mechanisms in MEET achieves promising data-driven learning of the temporal dependencies and spatial relationships of EEG signals across the entire brain in a holistic and comprehensive fashion.
Electroencephalogram (EEG) is one of the most widely used brain computer interface (BCI) approaches. Despite the success of existing EEG approaches in brain state recognition studies, it is still challenging to differentiate brain states via explainable and generalizable deep learning approaches. In other words, how to explore meaningful and distinguishing features and how to overcome the huge variability and overfitting problem still need to be further studied. To alleviate these challenges, in this work, a multiple random fragment search-based multilayer recurrent neural network (MRFS-MRNN) is proposed to improve the differentiating performance and explore meaningful patterns. Specifically, an explainable MRNN module is proposed to capture the temporal dependences preserved in EEG time series. Besides, a MRFS module is designed to cut multiple random fragments from the entire EEG signal time course to improve the effectiveness of brain state differentiating ability. MRFS-MRNN is concatenatedto effectively overcome the huge variabilities and overfitting problems. Experiment results demonstrate that the proposed MRFS-MRNN model not only has excellent differentiating performance, but also has good explanation and generalization ability. The classification accuracies reach as high as 95.18% for binary classification and 89.19% for four-category classification on the individual level. Similarly, 95.53% and 85.84% classification accuracies are obtained for the binary and four-category classification on the group level. What’s more, 94.28% and 85.43% classification accuracies of binary and four-category classifications are achieved for predicting brand new subjects. The experiment results showed that the proposed method outperformed other state-of-the-art (SOTA) models on the same underlying data and improved the explanation and generalization ability.
Large language models (LLMs) have undergone significant expansion and have been increasingly integrated across various domains. Notably, in the realm of robot task planning, LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions. However, for embodied tasks, where robots interact with complex environments, text-only LLMs often face challenges due to a lack of compatibility with robotic visual perception. This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks. Additionally, we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions. Our results, based on diverse datasets, indicate that GPT-4V effectively enhances robot performance in embodied tasks. This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights toward bridging the gap in Human-Robot-Environment interaction.
Brain–computer interface (BCI) technology enables humans to interact with computers by collecting and decoding electroencephalogram (EEG) from the brain. For practical BCIs based on EEG, accurate recognition is crucial. However, existing methods often struggle to achieve a balance between accuracy and complexity. To overcome these challenges, we propose 1D convolutional neural networks with bidirectional recurrent attention unit network (1DCNN-BiRAU) based on a random segment recombination strategy (segment pool, SegPool). It has three main contributions. First, SegPool is proposed to increase training data diversity and reduce the impact of a single splicing method on model performance across different tasks. Second, it employs multiple 1D CNNs, including local and global models, to extract channel information with simplicity and efficiency. Third, BiRAU is introduced to learn temporal information and identify key features in time-series data, using forward–backward networks and an attention gate in the RAU. The experiments show that our model is effective and robust, achieving accuracy of 99.47% and 91.21% in binary classification at the individual and group levels, and 90.90% and 92.18% in four-category classification. Our model demonstrates promising results for recognizing human motor imagery and has the potential to be applied in practical scenarios such as brain–computer interfaces and neurological disorder diagnosis.
Network topology construction is critical for designing efficient and resilient networks. Although ant colony optimization (ACO) has been widely applied in these tasks owing to its advantage of the pheromone update mechanism, it still exhibits limitations in solving such multi-objective optimization problems and converges slowly. This paper proposes a Diffusion-Based Multi-Objective Ant Colony Optimization (DMAC) algorithm for effective and reliable network topology construction. DMAC integrates pheromone diffusion mechanisms with multi-objective ACO to optimize multiple critical objectives concurrently and accelerate convergence. Experiments on simulation and real-world implementation demonstrate DMAC's ability to construct high-quality topologies balancing key tradeoffs within a small iteration budget. The proposed combination of diffusion mechanisms and multi-objective ACO addresses the limitations of prior ACO methods and provides new effective approach to automated network topology construction under multiple design constraints. DMAC demonstrates promising performance improvements on this complex multi-objective optimization problem and has great application potential in other fields and algorithms.