Intuitive interaction is the cornerstone of accurate and effective performance in remote robotic teleoperation. It requires high-fidelity in control actions as well as perception (vision, haptic, and other sensory feedback) of the remote environment. This paper presentsVicarios, a Virtual Reality (VR) based interface with the aim of facilitating intuitive real-time remote teleoperation, while utilizing the inherent benefits of VR, including immersive visualization, freedom of user viewpoint selection, and fluidity of interaction through natural action interfaces.Vicariosaims to enhance the situational awareness, using the concept ofviewpoint-independent mappingbetween the operator and the remote scene, thereby giving the operator better control in the perception-action loop. The article describes the overall system ofVicarios, with its software, hardware, and communication framework. A comparative user study quantifies the impact of the interface and its features, including immersion and instantaneous user viewpoint changes, termed “teleporting”, on users’ performance. The results show that users’ performance with the VR-based interface was either similar to or better than the baseline condition of traditional stereo video feedback, approving the realistic nature of theVicariosinterface. Furthermore, including the teleporting feature in VR significantly improved participants’ performance and their appreciation for it, which was evident in the post-questionnaire results.Vicarioscapitalizes on the intuitiveness and flexibility of VR to improve accuracy in remote teleoperation.
18 July 2022 in IEEE Robotics and Automation Letters
The current crisis surrounding the COVID-19 pandemic demonstrates the amount of responsibility and the workload on our healthcare system and, above all, on the medical staff around the world. In this work, we propose a promising approach to overcome this problem using robot-assisted telediagnostics, which allows medical experts to examine patients from distance. The designed telediagnostic system consists of two robotic arms. Each robot is located at the doctor and patient sites. Such a system enables the doctor to have a direct conversation via telepresence and to examine patients through robot-assisted inspection (guided tactile and audiovisual contact). The proposed bilateral teleoperation system is redundant in terms of teleoperation control algorithms and visual feedback. Specifically, we implemented two main control modes: joint-based and displacement-based teleoperation. The joint-based mode was implemented due to its high transparency and ease of mapping between Leader and Follower whereas the displacement-based is highly flexible in terms of relative pose mapping and null-space control. Tracking tests between Leader and Follower were conducted on our system using both wired and wireless connections. Moreover, our system was tested by seven medical doctors in two experiments. User studies demonstrated the system's usability and it was successfully validated by the medical experts.
25 October 2022 in International Journal of Social Robotics
Assistive social robots aim to facilitate outpatient-care including required safety critical measures. Accepting a robot to perform such measures, e.g., operate in close physical interaction for medical examinations, requires human trust towards the robot. Human-in-the-loop (HIL) applications where the robot is teleoperated by a human expert can help the person to accept even risky tasks performed by a robot. Therefore, the assistive humanoid GARMI was designed to enable HIL applications with varying autonomy. In this study, we use GARMI to understand which tasks in the framework of care may be accepted depending on human socio-demographics and user beliefs as well as the level of robot autonomy. Firstly, we seek to understand the general acceptance of GARMI using the Almere questionnaire. Secondly, we ask adults to rate their willingness to use several functionalities of GARMI. Lastly, we investigate the effect of the introduction method of GARMI on user acceptance. We assemble all relevant factors on acceptance to provide direction in the user-centered design process of assistive robots. The results of 166 participants show that alongside others, trust towards the robot and utilitarian variables such as perceived usefulness are the most influencing factors on the acceptance of GARMI and should be considered for the design of robotic semi-autonomous outpatient-services.
3 February 2021 in IEEE Robotics and Automation Letters
Humans use their limbs to perform various movements to interact with an external environment. Thanks to limb's variable and adaptive stiffness, humans can adapt their movements to the external unstable dynamics. The underlying adaptive mechanism has been investigated, employing a simple planar device perturbed by external 2D force patterns. In this work, we will employ a more advanced, compliant robot arm to extend previous work to a more realistic 3D-setting. We study the adaptive mechanism and use machine learning to capture the human adaptation behavior. In order to model human's stiffness adaptive skill, we give human subjects the task to reach for a target by moving a handle assembled on the end-effector of a compliant robotic arm. The arm is force controlled and the human is required to navigate the handle inside a non-visible, virtual maze and explore it only through robot force feedback when contacting maze virtual walls. By sampling the hand's position and force data, a computational model based on a combination of model predictive control and nonlinear regression is used to predict participants' successful trials. Our study shows that participants selectively increased the stiffness within the axis direction of uncertainty to compensate for instability caused by a divergent external force field. The learned controller was able to successfully mimic this behavior. When it is deployed on the robot for the navigation task, the robot arm successfully adapt to the unstable dynamics in the virtual maze, in a similar manner as observed in the participants’ adaptation skill.
4 October 2023 in IEEE Robotics and Automation Letters
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is well-established that humans exhibit extraordinary dexterity when handling objects. Such dexterity seems to derive from a robust understanding of object properties (such as weight, size, and shape), as well as a remarkable capacity to interact with them. Hand postures commonly demonstrate the influence of specific regions on objects that need to be grasped, especially when objects are partially visible. In this work, we leverage human-like object understanding by reconstructing and completing their full geometry from partial observations, and manipulating them using a 7-DoF anthropomorphic robot hand. Our approach has significantly improved the grasping success rates of baselines with only partial reconstruction by nearly 30% and achieved over 150 successful grasps with three different object categories. This demonstrates our approach's consistent ability to predict and execute grasping postures based on the completed object shapes from various directions and positions in real-world scenarios. Our work opens up new possibilities for enhancing robotic applications that require precise grasping and manipulation skills of real-world reconstructed objects.