Feature extraction and classification play an important role in brain computer interface (BCI) systems. In traditional approaches, methods in pattern recognition field are adopted to solve these problems. Nowadays, deep learning theory has developed so fast that researchers have employed it in many areas like computer vision and speech recognition, which have achieved remarkable results. However, few people introduce the deep learning method into the study of biomedical signals, especially EEG signals. In this study, a wavelet transform based input, which combines the time-frequency images of C3, Cz, and C4 channels, is proposed to extract the feature of motor imagery EEG signal. Then, a 2-Layer convolutional neural network is built as the classifier and convolutional kernels of different sizes are validated. The performance obtained by the proposed approach is evaluated by accuracy and Kappa value. The accuracy on dataset III from BCI competition II reaches 90%, and the best Kappa value on dataset 2a from competition IV is greater than many of other methods. In addition, the proposed method utilizes a resized small input, which reduces calculation complexity, so the training period is relatively faster. Results show that the method using convolutional neural network can be comparable or better than other state of art approaches, and the performance will be improved when there is sufficient data.

Efficiently identifying activities of daily living (ADL) provides very important contextual information that is able to improve the effectiveness of various sports tracking and healthcare applications. Recently, attention mechanism that selectively focuses on time series signals has been widely adopted in sensor based human activity recognition (HAR), which can enhance interesting target activity and ignore irrelevant background activity. Several attention mechanisms have been investigated, which achieve remarkable performance in HAR scenario. Despite their success, prior these attention methods ignore the cross-interaction between different dimensions. In the paper, in order to avoid above shortcoming, we present a triplet cross-dimension attention for sensor-based activity recognition task, where three attention branches are built to capture the cross-interaction between sensor dimension, temporal dimension and channel dimension. The effectiveness of triplet attention method is validated through extensive experiments on four public HAR dataset namely UCI-HAR, PAMAP2, WISDM and UNIMIB-SHAR as well as the weakly labeled HAR dataset. Extensive experiments show consistent improvements in classification performance with various backbone models such as plain CNN and ResNet, demonstrating a good generality ability of the triplet attention. Visualization analysis is provided to support our conclusion, and actual implementation is evaluated on a Raspberry Pi platform.

The issue of electroencephalogram (EEG)-based emotion recognition has great academic and practical significance. Currently, there are numerous research trying to address this issue in the literature. Particularly, transfer learning has gradually become a new methodological trend for the issue in company with the popularity of deep learning. Motivated by capturing the research panorama, summarizing the technological essence, and forecasting the advancement tendency of transfer learning for EEG-based emotion recognition, this article contributes a review work. This work mainly includes five aspects: 1) introducing the issue of EEG-based emotion recognition and expounding the importance of transfer learning for it; 2) analyzing the transfer learning framework and comparing it with the traditional ones; 3) elucidating the issue difficulties and explaining the suitability and capability of transfer learning for this issue; 4) summarizing, categorizing, and exemplifying the typical transfer learning methods for this issue; and 5) clarifying the methodological merits, discussing the challenging problems, and predicting the prospective development of transfer learning for the issue. We expect these contributions can inspire innovation and reformation of the transfer learning methodology for EEG-based emotion recognition as well as other relevant topics in the not-so-far future.

Federated Learning (FL) has recently attracted great interest in sensor-based human activity recognition (HAR) tasks. However, in real-world environment, sensor data on devices is non-independently and identically distributed (Non-IID), e.g., activity data recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, the traditional FL methods in the heterogeneous environment may incur a drifted global model that causes slow convergence and a heavy communication burden. Although some FL methods are gradually being applied to HAR, they are designed for overly ideal scenarios and do not address such Non-IID problem in the real-world setting. It is still a question whether they can be applied to cross-device FL. To tackle this challenge, we propose ProtoHAR, a prototype-guided FL framework for HAR, which aims to decouple the representation and classifier in the heterogeneous FL setting efficiently. It leverages the global prototype to correct the activity feature representation to make the prototype knowledge flow among clients without leaking privacy while solving a better classifier to avoid excessive drift of the local model in personalized training. Extensive experiments are conducted on four publicly available datasets: USC-HAD, UNIMIB-SHAR, PAMAP2, and HARBOX, which are collected in both controlled environments and real-world scenarios. The results show that compared with the state-of-the-art FL algorithms, ProtoHAR achieves the best performance and faster convergence speed in HAR datasets.

Recently, the state-of-the-art performance in various sensor-based human activity recognition (HAR) tasks has been acquired by deep learning, which can extract automatically features from raw data. In standard convolutional neural networks (CNNs), there is usually the same receptive field (RF) size of artificial neurons within each feature layer. It is well known that the RF size of neurons is able to change adaptively according to the stimulus, which has rarely been exploited in HAR. In this article, a new multibranch CNN is introduced, which utilizes a selective kernel mechanism for HAR. To the best of our knowledge, it is for the first time to adopt an attention idea to perform kernel selection among multiple branches with different RFs in the HAR scenario. We perform extensive experiments on several benchmark HAR datasets, namely, UCI-HAR, UNIMIB SHAR, WISDM, PAMAP2, and OPPORTUNITY, as well as weakly labeled datasets. Ablation experiments show that the selective kernel convolution can adaptively choose an appropriate RF size among multiple branches for classifying numerous human activities. As a result, it can achieve a higher recognition accuracy under a similar computing budget.