Benchmarks have long been used to verify and compare the readiness level of different technologies in many application domains. In the field of wearable robots, the lack of a recognized benchmarking methodology is one important impediment that may hamper the efficient translation of research prototypes into actual products. At the same time, an exponentially growing number of research studies are addressing the problem of quantifying the performance of robotic exoskeletons, resulting in a rich and highly heterogeneous picture of methods, variables and protocols. This review aims to organize this information, and identify the most promising performance indicators that can be converted into practical benchmarks. We focus our analysis on lower limb functions, including a wide spectrum of motor skills and performance indicators. We found that, in general, the evaluation of lower limb exoskeletons is still largely focused on straight walking, with poor coverage of most of the basic motor skills that make up the activities of daily life. Our analysis also reveals a clear bias towards generic kinematics and kinetic indicators, in spite of the metrics of human-robot interaction. Based on these results, we identify and discuss a number of promising research directions that may help the community to attain a comprehensive benchmarking methodology for robotassisted locomotion more efficiently.

Exoskeletons and exosuits have witnessed unprecedented growth in recent years, especially in the medical and industrial sectors. In order to be successfully integrated into the current society, these devices must comply with several commercialization rules and safety standards. Due to their intrinsic coupling with human limbs, one of the main challenges is to test and prove the quality of physical interaction with humans. However, the study of physical human–exoskeleton interactions (pHEI) has been poorly addressed in the literature. Understanding and identifying the technological ways to assess pHEI is necessary for the future acceptance and large-scale use of these devices. The harmonization of these evaluation processes represents a key factor in building a still missing accepted framework to inform human–device contact safety. In this review, we identify, analyze, and discuss the metrics, testing procedures, and measurement devices used to assess pHEI in the last ten years. Furthermore, we discuss the role of pHEI in safety contact evaluation. We found a very heterogeneous panorama in terms of sensors and testing methods, which are still far from considering realistic conditions and use-cases. We identified the main gaps and drawbacks of current approaches, pointing towards a number of promising research directions. This review aspires to help the wearable robotics community find agreements on interaction quality and safety assessment testing procedures.

Legged robotic technologies have moved out of the lab to operate in real environments, characterized by a wide variety of unpredictable irregularities and disturbances, all this in close proximity with humans. Demonstrating the ability of current robots to move robustly and reliably in these conditions is becoming essential to prove their safe operation. Here, we report an in-depth literature review aimed at verifying the existence of common or agreed protocols and metrics to test the performance of legged system in realistic environments. We primarily focused on three types of robotic technologies, i.e., hexapods, quadrupeds and bipeds. We also included a comprehensive overview on human locomotion studies, being it often considered the gold standard for performance, and one of the most important sources of bioinspiration for legged machines. We discovered that very few papers have rigorously studied robotic locomotion under irregular terrain conditions. On the contrary, numerous studies have addressed this problem on human gait, being nonetheless of highly heterogeneous nature in terms of experimental design. This lack of agreed methodology makes it challenging for the community to properly assess, compare and predict the performance of existing legged systems in real environments. On the one hand, this work provides a library of methods, metrics and experimental protocols, with a critical analysis on the limitations of the current approaches and future promising directions. On the other hand, it demonstrates the existence of an important lack of benchmarks in the literature, and the possibility of bridging different disciplines, e.g., the human and robotic, towards the definition of standardized procedures that will boost not only the scientific development of better bioinspired solutions, but also their market uptake.

The analysis of human locomotion is highly dependent on the quantity and quality of available data to obtain reliable evidence, due to the great variability of gait characteristics between subjects. Researchers usually have to make significant efforts to generate well-structured and trustworthy datasets. This situation is aggravated when patients are involved, due to experimental, privacy, and safety constraints. The availability of public datasets can facilitate this process. In this work, we systematically review the scientific and technical literature to identify the human locomotion databases publicly available nowadays. Within the 93 datasets identified, we observed that the most basic motor skills, e.g., flat or sloped walking, are well covered, whereas many other daily-life motor skills are poorly represented. The most common sensors used to record gait are optical motion capture systems, followed by RGB cameras and inertial sensors. We observed a lack of consistency in the data formats and limited sample size in most reviewed datasets. These issues hinder researchers from systematically standing on previous research results and represent a major barrier to using Artificial Intelligence and Big Data algorithms. With this work, we aim to provide the scientific community with a comprehensive, critical, and efficient guide to human locomotion datasets across different application domains.

Nowadays, robotic technology for gait training is becoming a common tool in rehabilitation hospitals. However, its effectiveness is still controversial. Traditional control strategies do not adequately integrate human intention and interaction and little is known regarding the impact of exoskeleton control strategies on muscle coordination, physical effort, and user acceptance. In this article, we benchmarked three types of exoskeleton control strategies in a sample of seven healthy volunteers: trajectory assistance (TC), compliant assistance (AC), and compliant assistance with EMG-Onset stepping control (OC), which allows the user to decide when to take a step during the walking cycle. This exploratory study was conducted within the EUROBENCH project facility. Experimental procedures and data analysis were conducted following EUROBENCH’s protocols. Specifically, exoskeleton kinematics, muscle activation, heart and breathing rates, skin conductance, as well as user-perceived effort were analyzed. Our results show that the OC controller showed robust performance in detecting stepping intention even using a corrupt EMG acquisition channel. The AC and OC controllers resulted in similar kinematic alterations compared to the TC controller. Muscle synergies remained similar to the synergies found in the literature, although some changes in muscle contribution were found, as well as an overall increase in agonist-antagonist co-contraction. The OC condition led to the decreased mean duration of activation of synergies. These differences were not reflected in the overall physiological impact of walking or subjective perception. We conclude that, although the AC and OC walking conditions allowed the users to modulate their walking pattern, the application of these two controllers did not translate into significant changes in the overall physiological cost of walking nor the perceived experience of use. Nonetheless, results suggest that both AC and OC controllers are potentially interesting approaches that can be explored as gait rehabilitation tools. Furthermore, the INTENTION project is, to our knowledge, the first study to benchmark the effects on human–exoskeleton interaction of three different exoskeleton controllers, including a new EMG-based controller designed by us and never tested in previous studies, which has made it possible to provide valuable third-party feedback on the use of the EUROBENCH facility and testbed, enriching the apprenticeship of the project consortium and contributing to the scientific community.