To enhance the adaptability and human-machine coordination of intelligent prosthetic knees, this study proposes a motion intention mapping direct myoelectric control method based on an LSTM network and a human-machine coupling model. Multichannel surface electromyography (sEMG) and knee joint angle data were collected during level-ground walking. Time-domain features were extracted to construct an LSTM prediction model, enabling temporal mapping between muscle activity and joint kinematics. Experimental results show that the LSTM model outperforms traditional neural networks in terms of prediction accuracy and temporal consistency. Furthermore, by integrating the human-machine coupling dynamics model with a hydraulic actuation system, a direct myoelectric control framework for a variable-damping prosthetic knee was established, achieving continuous damping adjustment and smooth gait transition. The results verify the feasibility and effectiveness of the proposed method in human-machine coordinated control.

Even as autonomous multi-robot systems (MRS) advance rapidly, current unmanned clusters struggle to complete many real-world robotic tasks relying solely on autonomous intelligence. Therefore, the human involvement is urgently required for MRS to effectively accomplish their missions in practical applications, and the hands-on control methods have been frequently employed for the operators to input MRS control commands. Whereas, in more complex environment where the tasks require both of the operator’s hands to be engaged, the hands-on approach is insufficient to achieve dependable human-MRS interaction. Therefore, this study represents an initial exploration into a hands-free control interface utilizing augmented reality (AR) and brain-computer interface (BCI) known as an AR-based FRP-BCI, an extension of the hands-on control approach. Specifically, the operator fixate on the target command menu displayed by Microsoft HoloLens2, the control command selection function is achieved with a more natural AR-based FRP-BCI via recognizing EEG signals. During the online experiment of the AR-based FRP-BCI command selection task with the MRS, the success rate of recognization is 90.67 ± 6.80 %. The proposed AR-based FRP-BCI system enables the users to select intention targets more natural in specific real-world environments. The results highlight the capability of the hands-free AR-based FRP-BCI to enhance traditional manual MRS input devices and the establishment of a more natural interface.

The head-mounted augmented reality (AR) device with gesture and verbal interfaces has been shown to facilitate co-located human-robot applications in dynamic environments. However, in hostile scenarios (e.g., search-and-rescue in after-disaster environment, reconnaissance in urban anti-terrorism, etc), the first-responder’s hands are often occupied with tasks and tools, leading to the demand for extra hands-free non-verbal instrumentation to detect and tag vital environment information (e.g., task-relevant targets) to robots. Recent studies on hands-free non-verbal instrumentation have elucidated several practical challenges, such as un-naturalistic efforts and constrained paradigms. Here, we proposed a wearable gaze-brain-computer interface (BCI) instrumentation that can directly detect and tag task-relevant targets based on the fixation-related potential (FRP) component of EEG signals, without flickering visual stimuli or limiting the position/duration of the observation. The proposed instrumentation was validated in a dual-task paradigm that simulated a real-world application, where the human has to concurrently perform knot tying with both hands and target detection/tagging with the proposed instrumentation. The results show that the wearable gaze-BCI instrumentation exhibited only marginal performance degradation in dual-tasking, with a decrease of 0.61 % (95 % CI: 0.10-1.11) in target classification accuracy (ACC), 1.00 bits/min (95 % CI: 0.13-1.88) in information transfer rate (ITR), and an increase of merely 0.39 % (95 % CI: -0.83-1.61) in false positive rate (FPR), compared to the single-tasking target detection. It suggests that the proposed instrumentation retains applicable in dual-tasking with high cognitive load demand. Moreover, the minor accuracy decrease of knot-tying task implies that target detection with the proposed instrumentation has little impact on accuracy of at-hand task. These findings reveal the potential of gaze-BCI as an extra hands-free non-verbal instrumentation for hands-occupied, while sustaining the performance of at-hand tasks.