Brain-computer interface (BCI) is a system empowering humans to communicate with or control the outside world with exclusively brain intentions. Electroencephalography (EEG)-based BCI is one of the promising solutions due to its convenient and portable instruments. Despite the extensive research of EEG in recent years, it is still challenging to interpret EEG signals effectively due to its nature of noise and difficulties in capturing the inconspicuous relations between EEG signals and specific brain activities. Most existing works either only consider EEG as chain-like sequences while neglecting complex dependencies between adjacent signals or requiring complex preprocessing. In this paper, we introduce two deep learning-based frameworks with novel spatio-temporal preserving representations of raw EEG streams to precisely identify human intentions. The two frameworks consist of both convolutional and recurrent neural networks effectively exploring the preserved spatial and temporal information in either a cascade or a parallel manner. Extensive experiments on a large scale movement intention EEG dataset (108 subjects, 3,145,160 EEG records) have demonstrated that the proposed frameworks achieve high accuracy of 98.3% and outperform a set of state-of-the-art and baseline models. The developed models are further evaluated with a real-world brain typing BCI and achieve a recognition accuracy of 93% over five instruction intentions suggesting good generalization over different kinds of intentions and BCI systems.

Recent years have witnessed the success of deep learning methods in human activity recognition (HAR). The longstanding shortage of labeled activity data inherently calls for a plethora of semisupervised learning methods, and one of the most challenging and common issues with semisupervised learning is the imbalanced distribution of labeled data over classes. Although the problem has long existed in broad real-world HAR applications, it is rarely explored in the literature. In this paper, we propose a semisupervised deep model for imbalanced activity recognition from multimodal wearable sensory data. We aim to address not only the challenges of multimodal sensor data (e.g., interperson variability and interclass similarity) but also the limited labeled data and class-imbalance issues simultaneously. In particular, we propose a pattern-balanced semisupervised framework to extract and preserve diverse latent patterns of activities. Furthermore, we exploit the independence of multi-modalities of sensory data and attentively identify salient regions that are indicative of human activities from inputs by our recurrent convolutional attention networks. Our experimental results demonstrate that the proposed model achieves a competitive performance compared to a multitude of state-of-the-art methods, both semisupervised and supervised ones, with 10% labeled training data. The results also show the robustness of our method over imbalanced, small training data sets.

The EEG signal is a medium to realize a BCI system due to its zero clinical risk and portable acquisition devices. Current EEG-based BCI research usually requires a subjectspecific adaptation step before a BCI can be employed by a new user. In contrast, the subject-independent scenario, where a well-trained model can be directly applied to new users without pre-calibration, is particularly desired. Considering this critical gap, the focus in this paper is developing an effective EEG signal analysis adaptively applied to subject-independent settings. We present a Convolutional Recurrent Attention Model (CRAM) that utilizes a convolutional neural network to encode the highlevel representation of EEG signals and a recurrent attention mechanism to explore the temporal dynamics of the EEG signals as well as to focus on the most discriminative temporal periods. Extensive experiments on a benchmark multiclass EEG dataset containing four movement intentions indicate that the proposed model is capable of exploiting the underlying invariant EEG patterns across different subjects and generalizing the patterns to new subjects with better performance than a series of state-ofthe-art and baseline approaches by at least 8 percentage points. The implementation code is made publicly available.

Motor imagery classification from EEG signals is essential for motor rehabilitation with a brain-computer interface (BCI). Most current works on this issue require a subject-specific adaptation step before applying a BCI to a new user. Thus the research of directly extending a pre-trained model to new users is particularly desired and indispensable. As brain dynamics fluctuate considerably across different subjects, it is challenging to design practical hand-crafted features based on prior knowledge. Regarding this gap, this paper proposes a Graph-based Convolutional Recurrent Attention Model (G-CRAM) to explore EEG features across different subjects for motor imagery classification. A graph structure is first developed to represent the positioning information of EEG nodes. Then a convolutional recurrent attention model learns EEG features from both spatial and temporal dimensions and emphasizes the most distinguishable temporal periods. We evaluate the proposed approach on two benchmark EEG datasets for motor imagery classification on the subject-independent testing. The results show that the G-CRAM achieves superior performance to state-of-the-art methods regarding recognition accuracy and ROC-AUC. Furthermore, model interpreting studies reveal the learning process of different neural network components and demonstrate that the proposed model can extract detailed features efficiently.

An EEG-based Brain-Computer Interface (BCI) is a system that enables a user to communicate with and intuitively control external devices solely using the user’s intentions. Current EEG-based BCI research usually involves a subject-specific adaptation step before a BCI system is ready to be employed by a new user. However, the subject-independent scenario, in which a well-trained model can be directly applied to new users without pre-calibration, is particularly desirable yet rarely explored. Considering this critical gap, our focus in this paper is the subject-independent scenario of EEG-based human intention recognition. We present a Graph-based Hierarchical Attention Model (G-HAM) that utilizes the graph structure to represent the spatial information of EEG sensors and the hierarchical attention mechanism to focus on both the most discriminative temporal periods and EEG nodes. Extensive experiments on a large EEG dataset containing 105 subjects indicate that our model is capable of exploiting the underlying invariant EEG patterns across different subjects and generalizing the patterns to new subjects with better performance than a series of state-of-the-art and baseline approaches.

Trajectory similarity computation serves as a fundamental functionality of various spatial information applications. Although existing deep learning similarity computation methods offer better efficiency and accuracy than non-learning solutions, they are still immature in trajectory embedding and suffer from poor generality and heavy preprocessing for training. Targeting these limitations, we propose a novel framework named KGTS based on knowledge graph grid embedding, prompt trajectory embedding, and unsupervised contrastive learning for improved trajectory similarity computation. Specifically, we first embed map grids with a GRot embedding method to vigorously grasp the neighbouring relations of grids. Then, a prompt trajectory embedding network incorporates the resulting grid embedding and extracts trajectory structure and point order information. It is trained by unsupervised contrastive learning, which not only alleviates the heavy preprocessing burden but also provides exceptional generality with creatively designed strategies for positive sample generation. The prompt trajectory embedding adopts a customized prompt paradigm to mitigate the gap between the grid embedding and the trajectory embedding. Extensive experiments on two real-world trajectory datasets demonstrate the superior performance of KGTS over state-of-the-art methods.