1 June 2022 in IEEE Robotics and Automation Letters
This study aims to demonstrate reinforcement learning tracking control for automatically configuring the impedance parameters of a robotic knee prosthesis. While our previous studies involving human subjects have focused on tuning the impedance control parameters to meet a fixed, subjectively prescribed target motion profile to enable continuous walking with human-in-the-loop, in this paper we develop a new tracking control solution for a robotic knee to mimic the motion of the intact knee. As such, we replaced the prescribed target knee motion by an automatically generated profile based on the intact knee. As the profile of the intact knee varies over time due to human adaptation, we are presented with a challenging tracking control problem in the context of classical control theory. By formulating the “echo control” of the robotic knee as a reinforcement learning problem, we provide a promising new tool for real-time tracking control design without explicitly representing the underlying dynamics using a mathematical model, which can be difficult to obtain for a human-robot system. Additionally, our results may inspire future studies and new robotic prosthesis impedance control designs that can potentially coordinate between the intact and the robotic limbs toward daily use of the robotic device.
25 October 2023 in Science Robotics
Current lower-limb prostheses do not provide active assistance in postural control tasks to maintain the user’s balance, particularly in situations of perturbation. In this study, we aimed to address this missing function by enabling neural control of robotic lower-limb prostheses. Specifically, electromyographic (EMG) signals (amplified neural control signals) recorded from antagonistic residual ankle muscles were used to drive a robotic prosthetic ankle directly and continuously. Participants with transtibial amputation were recruited and trained in using the EMG-driven robotic ankle. We studied how using the EMG-controlled ankle affected the participants’ anticipatory and compensatory postural control strategies and stability under expected perturbations compared with using their daily passive devices. We investigated the similarity of neuromuscular coordination (by analyzing motor modules) of the participants, using either device in a postural sway task, to that of able-bodied controls. Results showed that, compared with their passive prosthesis, the EMG-controlled prosthesis enabled participants to use near-normative postural control strategies, as evidenced by improved between-limb symmetry in intact-prosthetic center-of-pressure and joint angle excursions. Participants substantially improved postural stability, as evidenced by a reduction in steps or falls using the EMG-controlled prosthetic ankle. Furthermore, after relearning to use residual ankle muscles to drive the robotic ankle in postural control, nearly all participants’ motor module structure shifted toward that observed in individuals without limb amputations. Here, we have demonstrated the potential benefit of direct EMG control of robotic lower limb prostheses to restore normative postural control strategies (both neural and biomechanical) toward enhancing standing postural stability in amputee users.
18 January 2022 in IEEE Robotics and Automation Letters
Quantitatively characterizing a locomotion performance objective for a human-robot system is an important consideration in the assistive wearable robot design towards human-robot symbiosis. This problem, however, has only been addressed sparsely in the literature. In this study, we propose a new inverse approach from observed human-robot walking behavior to infer a human-robot collective performance objective represented in a quadratic form. By an innovative design of human experiments and simulation study, respectively, we validated the effectiveness of two solution approaches to solving the inverse problem using inverse reinforcement learning (IRL) and inverse optimal control (IOC). The IRL-based experiments of human walking with robotic transfemoral prosthesis validated the realistic applicability of the proposed inverse approach, while the IOC-based analysis provided important human-robot system properties such as stability and robustness that are difficult to obtain from human experiments. This study introduces a new tool to the field of wearable lower limb robots. It is expected to be expandable to quantify joint human-robot locomotion performance objectives for personalizing wearable robot control in the future.
27 July 2022 in IEEE Robotics and Automation Letters
Recent efforts in the design of intelligent controllers for configuring robotic prostheses have demonstrated new possibilities in improving mobility and restoring locomotion for individuals with lower-limb disabilities. In these efforts, personalizing the controller of the robotic device is a crucial step in order to meet individual user's needs and physical conditions. Reinforcement learning (RL) based control designs are among some of the most promising approaches to achieving real-time, optimal adaptive tuning capability. However, such designs to date rely on subjectively determining human-robot walking performance measures, commonly in a quadratic form. To further automate the RL design for robotic knee control parameter tuning and potentially improve human-robot locomotion performance, this study introduces a new bilevel optimization method to objectively specify such control design performance measures via inverse reinforcement learning (IRL), which in turn, will be used in low level (forward) RL design of the impedance control parameters. We demonstrate the effectiveness of the bilevel optimization approach with improved human-robot walking performance using systematic OpenSim simulation studies.
27 June 2022 in IEEE Robotics and Automation Letters
The physical interactions between wearable lower limb robots and humans have been investigated to inform effective robot design for walking augmentation. However, human-robot interactions when internal faults occur within robots have not been systematically reported, but it is essential to improve the robustness of robotic devices and ensure the user’s safety. This letter aims to (1) present a methodology to characterize the behavior of the robotic transfemoral prosthesis as an effective wearable robot platform while interacting with the users in the presence of internal faults, and (2) identify the potential data sources for accurate detection of the prosthesis fault. We first obtained the human perceived response in terms of their walking stability when the prosthesis control fault (inappropriate intrinsic control output/command) was emulated/applied in level-ground walking. Then the measurements and their features, obtained from the transfemoral prosthesis, were examined for the emulated faults that elicited a sense of instability in human users. The optimal features that contributed the most in separating faulty interaction from the normal walking condition were determined using two machine-learning-based approaches: One-Class Support Vector Machine (OCSVM) and Mahalanobis Distance (MD) classifier. The OCSVM anomaly detector could achieve an average sensitivity of 85.7% and an average false alarm rate of 1.7% with a reasonable detecting time of 147.6 ms for detecting emulated control errors among all subjects. The result demonstrates the potential of using machine-learning-based schemes in identifying prosthesis control faults based on intrinsic sensors on the prosthesis. This study presents a procedure to study human-robot fault tolerance and inform the future design of robust prosthesis control.
23 November 2022 in IEEE Transactions on Biomedical Engineering
Automatically personalizing complex control of robotic prostheses to improve gait performance, such as gait symmetry, is challenging. Recently, human-in-the-loop (HIL) optimization and reinforcement learning (RL) have shown promise in achieving optimized control of wearable robots for each individual user. However, HIL optimization methods lack scalability for high-dimensional space, while RL has mostly focused on optimizing robot kinematic performance. Thus, we propose a novel hierarchical framework to personalize robotic knee prosthesis control and improve overall gait performance. Specifically, in this study the framework was implemented to simultaneously design target knee kinematics and tune 12 impedance control parameters for improved symmetry of propulsive impulse in walking. In our proposed framework, HIL optimization is used to identify an optimal target knee kinematics with respect to symmetry improvement, while RL is leveraged to yield an optimal policy for tuning impedance parameters in high-dimensional space to match the kinematics target. The proposed framework was validated on human subjects, walking with robotic knee prosthesis. The results showed that our design successfully shaped the target knee kinematics as well as configured 12 impedance control parameters to improve propulsive impulse symmetry of the human users. The knee kinematics that yielded best propulsion symmetry did not preserve the normative knee kinematics profile observed in non-disabled individuals, suggesting that restoration of normative joint biomechanics in walking does not necessarily optimize the gait performance of human-prosthesis systems. This new framework for prosthesis control personalization may be extended to other wearable devices or different gait performance optimization goals in the future.