In this article, we develop a robust sliding-mode nonlinear predictive controller for brain-controlled robots with enhanced performance, safety, and robustness. First, the kinematics and dynamics of a mobile robot are built. After that, the proposed controller is developed by cascading a predictive controller and a smooth sliding-mode controller. The predictive controller integrates the human intention tracking with safety guarantee objectives into an optimization problem to minimize the invasion to human intention while maintaining robot safety. The smooth sliding-mode controller is designed to achieve robust desired velocity tracking. The results of human-in-the-loop simulation and robotic experiments both show the efficacy and robust performance of the proposed controller. This work provides an enabling design to enhance the future research and development of brain-controlled robots.

Using brain signals rather than limbs to drive a vehicle can help persons with disabilities to extend their movement range and, thus, to improve their self-independence. However, the driving performance of brain-controlled vehicles (BCVs) is poor. In this paper, to improve the performance of BCVs, we propose a new shared control method based on the model predictive control (MPC) strategy. Particularly, to maintain the maximum control authority of brain-control drivers while ensuring the safety of BCVs, the MPC controller is designed by introducing a penalty on the deviation from drivers output in the cost function and setting safety constraints. Driver-and-hardware-in-the-loop experiments are conducted under two road-keeping scenarios and one obstacle-avoidance scenario with different subjects to validate the proposed method. The results demonstrate the effectiveness of the proposed method in avoiding roadway departures and obstacles while maintaining the control authority of users.

The driver’s state and behavior are crucial for the driving process, which affect the driving safety directly or indirectly. Electroencephalography (EEG) signals have the advantage of predictability and have been widely used to detect and predict the users’ states and behaviors. Accordingly, the EEG-based driver state and behavior detection, which can be integrated into the intelligent vehicles, is becoming the hot research topic to develop an intelligent assisted driving system (IADS). In this paper, we systematically reviewed the EEG-based driver state and behavior detection for intelligent vehicles. First, we concluded the most popular methods for EEG-based IADS, including the algorithms of the signal acquisition, preprocessing, signal enhancement, feature calculation, feature selection, classification, and post-processing. Then, we surveyed the research on separate EEG-based driver state detection and the driver behavior detection, respectively. The research on EEG-based combinations of driver state and behavior detection was further reviewed. For the review of these studies of driver state, behavior, and combined state and behavior, we not only defined the related fundamental information and overviewed the research on single EEG-based brain-computer interface (BCI) applications, but also further explored the relevant research progress on the EEG-based hybrid BCIs. Finally, we thoroughly discussed the current challenges, possible solutions, and future research directions.

Image compressed sensing (ICS) has been extensively applied in various imaging domains due to its capability to sample and reconstruct images at subNyquist sampling rates. The current predominant approaches in ICS, specifically pure convolutional networks (ConvNets)-based ICS methods, have demonstrated their effectiveness in capturing local features for image recovery. Simultaneously, the Transformer architecture has gained significant attention due to its capability to model global correlations among image features. Motivated by these insights, we propose a novel hybrid network for ICS, named MTC-CSNet, which effectively combines the strengths of both ConvNets and Transformer architectures in capturing local and global image features to achieve high-quality image recovery. Particularly, MTC-CSNet is a dual-path framework that consists of a ConvNets-based recovery branch and a Transformer-based recovery branch. Along the ConvNets-based recovery branch, we design a lightweight scheme to capture the local features in natural images. Meanwhile, we implement a Transformer-based recovery branch to iteratively model the global dependencies among image patches. Ultimately, the ConvNets-based and Transformer-based recovery branches collaborate through a bridging unit, facilitating the adaptive transmission and fusion of informative features for image reconstruction. Extensive experimental results demonstrate that our proposed MTC-CSNet surpasses the state-of-the-art methods on various public datasets. The code and models are publicly available at MTC-CSNet.

Data glove is one of the devices that can express human intention via detecting hand motion with high-precision. In recent years, flourished applications have been driven by data gloves from human-machine interaction (HMI) to biomedical sciences. To understand the latest research status and grasp the development direction, this study provides a systematic overview of the state-of-the-art for data gloves. Specifically, we take the research topics of hardware design, algorithms for gesture recognition, and the application areas as the entry point. Detailed elaboration of key algorithms techniques, from data processing to classification, for hand gesture recognition is made and quantitative analysis of the recent research achievements is summarized. We found that hand motion detection with data gloves is mostly employed for gesture recognition, and the majority of studies are focused on the performance validation of the gesture recognition, in particular for static gestures. A systematic illustration of the current research challenges as well as the future directions of related data gloves are finally discussed.

Brain-controlled mobile robotics can provide restoration of mobility for individuals with severe physical disabilities and empower healthy people with a broader reachable range in particular environments, which have been flourished over the past twenty years. This paper conducts a systematic state-of-the-art overview of noninvasive EEG-based intelligent mobile robots. We first present the general architecture and basic concepts, typical system types, and main research efforts on the whole-system design. Then, relevant key techniques associated with the brain-machine interfaces (BMIs), control strategies, and robot intelligence are reviewed to elucidate the research progress of the overall system. System performance evaluation is critical and complicated, here we summarize the conditions of the recruited participants, the experimental protocol, tasks and environments, with an emphasis on evaluation metrics regarding BMI performance, navigation performance, system robustness, and the user. We further highlight the remaining challenges and the potential research directions of future work. This study with informative outline is envisioned to enhance current understanding and suggest the future perspectives on EEG-based mobile robotic devices Note to Practitioners —The brain-computer interfaces (BCIs) can provide a novel and feasible way to convey the users’ intentions to the terminal devices through the brain signals, which has exhibited emerging prospects in both civil and military applications thus having attracted increasing interests in researchers, industries, and government. The integration of noninvasive electroencephalogram (EEG)-based BCIs and expanded wheeled robotics have potentials to improve the human daily life with enhanced level of mobility, exhibiting significant socioeconomic impacts. This paper provides a systematic review of the EEG-based intelligent mobile robots from the aspects of key milestones, key techniques, and performance evaluation. To bring such the next generation assistive products from the laboratory to real applications, this paper further highlights the remaining challenges and shed light on the potential future work.