Current EEG-based brain-computer interface technologies mainly focus on how to independently use SSVEP, motor imagery, P300, or other signals to recognize human intention and generate several control commands. SSVEP and P300 require external stimulus, while motor imagery does not require it. However, the generated control commands of these methods are limited and cannot control a robot to provide satisfactory service to the user. Taking advantage of both SSVEP and motor imagery, this paper aims to design a hybrid BCI system that can provide multimodal BCI control commands to the robot. In this hybrid BCI system, three SSVEP signals are used to control the robot to move forward, turn left, and turn right; one motor imagery signal is used to control the robot to execute the grasp motion. In order to enhance the performance of the hybrid BCI system, a visual servo module is also developed to control the robot to execute the grasp task. The effect of the entire system is verified in a simulation platform and a real humanoid robot, respectively. The experimental results show that all of the subjects were able to successfully use this hybrid BCI system with relative ease.

Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players’ different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.

Automatic driving vehicles (ADV) have been developed to provide more convenient and comfortable driving experiences. However, these vehicles failed in satisfying the variance of human intentions. Recently, the strategy of collaborating brain-computer interface (BCI) controlling and automatic driving receives attention. Since the BCI system remained some limitation in real-time controlling, a fusion method has been proposed to explore and verify the feasibility of human-vehicle collaborative driving (HVCD) in this work. A hybrid brain-computer interface was developed to interpret human intentions. In addition, a computer vision-based automatic driving component was developed to maintain the vehicle on the road. A system for fusing these two kinds of vehicle driving decisions was firstly proposed in this paper. This system can simultaneously obtain the visual data and the hybrid electroencephalograph (EEG) signals. The hybrid EEG signals consist of steady-state visual evoked potentials (SSVEP) and motor imagery (MI). The obtained multi-source information can be fused to make the final decision to drive a simulated vehicle. The proposed system was evaluated with different destinations. The experimental results verify the feasibility of fusing both human intention and computer vision. The task success rate reached 91.1% and the information transfer rate (ITR) was 85.80 bit/min.

Semi-global bipartite fault-tolerant containment control framework on antagonistic communication networks is proposed in this article for heterogeneous multiagent systems (MASs) under the influence of input saturation and actuator faults. An observer is constructed to estimate the leaders’ states on signed digraph, where the communication networks are antagonistic. A fully distributed virtual control approach is developed to acquire the containment trajectory. Based on the observer, a semi-global containment control method is developed to compensate for the detrimental impacts of both input saturation and actuator faults. Besides, the dynamics and state-space dimensions of the agents can be different. The proposed framework overcomes two drawbacks of the conventional containment control: 1) the containment trajectory is obtained under general antagonistic communication networks, which is more general in engineering applications and 2) both actuator faults and input saturation are solved for heterogeneous agents, which relaxes the limitation of homogeneous dynamics. Finally, a simulation example is conducted to test and verify the feasibility of the proposed method framework.

Human-vehicle collaborative driving (HVCD) as a novel concept of intermediate stage from the development of normal smart vehicle to self-driving vehicle, requires the vehicle’s decisionmaking authority equivalent to the human driver’s operation. As the self-driving vehicle cannot achieve full-condition driving without human driver yet and still have a long time to go, the collaborative driving can exploit both machine and human specialties, which requires human and vehicle intelligence to control the vehicle simultaneously and work together. However, there were few studies focusing on the HVCD that experimental environment and evaluation methods are rarely discussed, even under the simulation conditions, which limits the development of collaborative driving. To bridge the significant gaps, this paper designed a simple semi-physical simulation system architecture for the HVCD. First, the simple designed system can achieve basic functions, including human control input as normal driver, vehicle intelligence with local and global sensor data process and decision fusion. The presented system can use and process these multi-source information to achieve the HVCD in the simulated environment. Second, evaluation method based on vehicle driving safety, stability, and rapidity are proposed and discussed, which matters most for system performance test and feasibility verification. In addition, experiments with different decision fusion conditions were presented and the experimental results show that the proposed simple semi-physical HVCD simulation system can work smoothly and reflect the practical scenario. Furthermore, the performance of different fusion weights experiments shows the advantages of both human driver and vehicle intelligence, which provides a reference for the HVCD related research improvement.