Brain–computer interface (BCI) is a typical direction of integration of human intelligence and robot intelligence. Shared control is an essential form of combining human and robot agents in a common task, but still faces a lack of freedom for the human agent. This paper proposes a Centroidal Voronoi Tessellation (CVT)-based road segmentation approach for brain-controlled robot navigation by means of asynchronous BCI. An electromyogram-based asynchronous mechanism is introduced into the BCI system for self-paced control. A novel CVT-based road segmentation method is provided to generate optional navigation goals in the road area for arbitrary goal selection. An event-related potential of the BCI is designed for target selection to communicate with the robot. The robot has an autonomous navigation function to reach the human selected goals. A comparison experiment in the single-step control pattern is executed to verify the effectiveness of the CVT-based asynchronous (CVT-A) BCI system. Eight subjects participated in the experiment, and they were instructed to control the robot to navigate toward a destination with obstacle avoidance tasks. The results show that the CVT-A BCI system can shorten the task duration, decrease the command times, and optimize navigation path, compared with the single-step pattern. Moreover, this shared control mechanism of the CVT-A BCI system contributes to the promotion of human and robot agent integration control in unstructured environments.

Brain-computer interface (BCI) is a useful device for people without relying on peripheral nerves and muscles. However, the performance of the event-related potential (ERP)-based BCI declines when applying it to real environments, especially in cross-state and cross-subject conditions. Here we employ temporal modeling and adversarial training to improve the visual ERP-based BCI under different mental workload states and to alleviate the problems above. The rationality of our method is that the ERP-based BCI is based on electroencephalography (EEG) signals recorded from the scalp’s surface, continuously changing with time and somewhat stochastic. In this paper, we propose a hierarchical recurrent network to encode all ERP signals in each repetition at the same time and model them with a temporal manner to predict which visual event elicited an ERP. The hierarchical architecture is a simple yet effective method for organizing recurrent layers in a deep structure to model long sequence signals. Taking a cue from recent advances in adversarial training, we further applied dynamic adversarial perturbations to create adversarial examples to enhance the model performance. We conduct our experiments on one published visual ERP-based BCI task with 15 subjects and 3 different auditory workload states. The results indicate that our hierarchical method can effectively model the long sequence EEG raw data, outperform the baselines on most conditions, including cross-state and cross-subject conditions. Finally, we show how deep learning-based methods with limited EEG data can improve ERP-based BCI with adversarial training. Our code is available at https://github.com/aispeech-lab/VisBCI .

The performance of the event-related potential (ERP)-based brain–computer interface (BCI) declines when applying it into the real environment, which limits the generality of the BCI. The sound is a common noise in daily life, and whether it has influence on this decline is unknown. This study designs a visual-auditory BCI task that requires the subject to focus on the visual interface to output commands and simultaneously count number according to an auditory story. The story is played at three speeds to cause different workloads. Data collected under the same or different workloads are used to train and test classifiers. The results show that when the speed of playing the story increases, the amplitudes of P300 and N200 potentials decrease by 0.86 μV (p = 0.0239) and 0.69 μV (p = 0.0158) in occipital-parietal area, leading to a 5.95% decline (p = 0.0101) of accuracy and 9.53 bits/min decline (p = 0.0416) of information transfer rate. The classifier that is trained by the high workload data achieves higher accuracy than the one trained by the low workload if using the high workload data to test the performance. The result indicates that the sound could affect the visual ERP-BCI by increasing the workload. The large similarity of the training data and testing data is as important as the amplitudes of the ERP on obtaining high performance, which gives us an insight on how make to the ERP-BCI generalized.