13 July 2022 in ACM Transactions on Human-Robot Interaction
We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provide a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 envisions cooperation between human and robot working in teams. Indeed, there exist many industrial tasks that can benefit from human-robot collaboration. A prime example is high-value composite manufacturing. Working with our industry partner towards this example application, we evaluated our AR interface design for shared physical workspace collaboration in human-robot teams. We conducted a multi-dimensional analysis of our interface using established metrics. Results from our user study (n = 26) show that, subjectively, the AR interface feels more novel and a standard joystick interface feels more dependable to users. However, the AR interface was found to reduce physical demand and task completion time, while increasing robot utilization. Furthermore, user’s freedom of choice to collaborate with the robot may also affect the perceived usability of the system.
18 July 2022 in IEEE Robotics and Automation Letters
HLH. F. Machiel Van der Loos
H. F. Machiel Van der Loos
Learning from Demonstration (LfD) seeks to democratize robotics by enabling non-expert end-users to teach robots. However, most LfD techniques assume users provide optimal demonstrations, which may not be accurate. Demonstration quality plays a crucial role in robot learning and generalization. Hence, it is important to quantify the quality of the provided demonstrations before using them for robot learning. In this letter, we propose quantifying the generalizability of the demonstrations based on how well they perform in the learned task. The proposed approach is validated in a user study (N = 27). Participants with different robotics expertise levels were recruited to teach a PR2 robot a generic task (pressing a button) under different task constraints. They taught the robot in two sessions on two different days to capture their teaching behaviour across sessions. The task performance was utilized to classify the provided demonstrations into high-quality and low-quality sets. The results show a significant correlation between task performance and generalization performance across all participants. We also found that users clustered into two groups: Users who provided high-quality demonstrations from the first session (the fast-adapters ), and users who provided low-quality demonstrations in the first session and then improved with practice (the slow-adapters ). This approach for assessing demonstrations allows us to determine whether users require more training in order to provide high-quality demonstrations.
28 December 2018 in IEEE Robotics and Automation Letters
In this paper, we propose a training approach combining hand-over-hand and trial and error training approaches and we evaluate its effectiveness for both robotic and standard laparoscopic surgical training. The proposed approach makes use of the data of an expert collected while using the da Vinci Surgical System. We present our data collection system and how we use it in the proposed training approach. We conduct two user studies (N = 21 for each) to evaluate the effectiveness of this approach. Our results show that subjects trained using this combined approach can better balance the speed and accuracy of their task execution compared with others trained using only one of either hand-over-hand or trial and error training approaches. Moreover, this combined approach leads to the best performance when it comes to the transferability of the acquired skills when testing on another task. We show that the results of the two studies are consistent with an established model in the literature for motor skill learning. Moreover, our results show for the first time the feasibility of using a surgical robot and data collected from it as a training platform for conventional laparoscopic surgery without robot assistance.
1 January 2021 in Annual Reviews in Control
20 August 2025 in ACM Transactions on Human-Robot Interaction
Learning from Demonstration (LfD) is a framework that allows lay users to easily program robots. However, the efficiency of robot learning and the robot’s ability to generalize to task variations hinge upon the quality and quantity of the provided demonstrations. Our objective is to guide human teachers to provide more effective demonstrations, thus facilitating efficient robot learning. To achieve this, we propose to use a measure of uncertainty, namely task-related information entropy , as a criterion for suggesting informative demonstration examples to human teachers to improve their teaching skills. This approach seeks to minimize the requisite number of demonstrations by enhancing their distribution throughout the workspace. In a conducted experiment
\((N = 24)\)
, an augmented reality (AR)-based guidance system was employed to train novice users to produce additional demonstrations from areas with the highest entropy within the workspace. These novice users were trained for a few trials to teach the robot a generalizable task using a limited number of demonstrations. Subsequently, the users’ performance after training was assessed first on the same task (retention) and then on a new task (transfer) without guidance. The results indicate a substantial improvement in robot learning efficiency from the teacher’s demonstrations, with an improvement of up to 198% observed on the novel task. Furthermore, the proposed approach was compared to a state-of-the-art heuristic rule and found to improve robot learning efficiency by 210% compared to the heuristic rule. The scripts used in this article are available on GitHub .