Large Language Models (LLMs) are gaining popularity in the field of robotics. However, LLM-based robots are limited to simple, repetitive motions due to the poor integration between language models, robots, and the environment. This letter proposes a novel approach to enhance the performance of LLM-based autonomous manipulation through Human-Robot Collaboration (HRC). The approach involves using a prompted GPT-4 language model to decompose high-level language commands into sequences of motions that can be executed by the robot. The system also employs a YOLO-based perception algorithm, providing visual cues to the LLM, which aids in planning feasible motions within the specific environment. Additionally, an HRC method is proposed by combining teleoperation and Dynamic Movement Primitives (DMP), allowing the LLM-based robot to learn from human guidance. Real-world experiments have been conducted using the Toyota Human Support Robot for manipulation tasks. The outcomes indicate that tasks requiring complex trajectory planning and reasoning over environments can be efficiently accomplished through the incorporation of human demonstrations.

Remote teleoperation has shown significant advancements since the first teleoperation system was proposed by Goertz in the 1940s. In recent years, the research on shared control methodologies in which the robot assists the operators in accomplishing the desired tasks has gained extensive attention. One such important task in teleoperation is object grasping. In this paper, we propose a shared control framework to enhance the teleoperated grasping performance. The proposed framework is built upon a virtual reality device-based direct teleoperation system. In this framework, a template matching-based object point cloud compensation is introduced for multi-angle grasping pose generation. Then, the feasible grasping candidates are selected considering joint constraints-aware manipulability. Finally, the grasping assistance is achieved by trajectory blending with dynamic authority adjustment. To validate the performance of the proposed framework, we carried out experimental evaluations. The output results indicate improved grasping performance in terms of reduced task completion time, linear trajectory, and workload.

Robot-assisted minimally invasive surgery (RMIS) has been shown to be effective in improving surgeon capabilities, providing magnified 3D vision, highly dexterous surgical tools, and intuitive human-robot interfaces for high-precision tool motion control. Robotic surgical tools (RST) are a critical component that defines the performance of an RMIS system. Current RSTs still represent a high cost, with few commercially available options, which limits general access and research on RMIS. We aim to take advantage of recent progress in biocompatible 3D printing and contribute to the development of RMIS technologies, presenting an open platform for low-cost, biocompatible, and customizable RSTs. The proposed design concept consists of a 3-DOF end-effector with a decoupled wrist mechanism, a tool interface module, and a tool drive unit. We validated our end-effector design using Finite Element Analysis (FEA) to confirm that stress generated by high grip forces is maintained below the material yield stress. Validation experiments showed that the proposed RST could provide up to 10N grip forces and up to 3N pulling forces. The proposed control framework exhibited a mean absolute positioning tracking error of approximately 0.1 rad. Finally, we also demonstrated the use of the proposed RST in two surgical training tasks: pick-and-place and stitching. The designs and software control framework are open-access and freely available for customization and fast development at https://github.com/jcolan/OpenRST.

Robotic teleoperation with force feedback has been studied extensively since it was first developed in the 1940s. Time delay is a common problem of bilateral teleoperation systems. Although many efforts on optimizing the control architectures have been made, there is always a trade-off between transparency and stability for bilateral systems, and the perfect transparency and stability can only be achieved simultaneously in ideal situations. In this paper, we propose a novel approach to compensate for the degraded transparency while using the conventional passivity-based approach to maintain system stability under time-delay. The proposed approach is based on visual feedback and enhances the transparency by displaying different kinds of onomatopoeia according to contact force detected on the slave side. The basic performance is evaluated by conducting a stiffness classification task under constant round trip time delays (0 ms, 500 ms and 1000 ms). The result indicates that the subject has higher accuracy for classifying the stiffness of a remote object by using onomatopoeia enhanced force feedback compared with the conventional passivity-based position-force feedback.

Recently, wearable extra robotic limbs that aim to enhance the functionality and capability of human operators as extra arms or fingers have become an active research subject among robotics researchers. Improving the operability of the extra robotic limbs is required for the human operators, an approach for which is to induce robotic embodiment. In this paper, we focus on the update of sense of self-location which is the key aspect of embodiment and contributes to the body representation update, and we elucidate dominant factors which induce the embodiment of an extra robotic thumb (ERT). The experiments are conducted to compare the performance of the reaching task of the ERT under three separately given conditions, of which two are somatosensory feedbacks: (1) tactile and position feedback from the fingertips and (2) tactile and position feedback from the human face, and one is a non-somatosensory feedback: (3) auditory feedback. As a result, we confirmed that the somatosensory feedback from finger tips has a large contribution for the update of sense of self-location.

Wearable robotic limbs support sit-to-stand (STS) transfer with increased stability while maintaining a compact size, as well as providing body weight support. This paper proposes a new robotic limb, Variable Grounding Flexible Limb (VGFL). The VGFL achieved a novel strategy in which its grounding point tracks the forward shift of the wearer's center of gravity (CoG) during STS. Since the strategy keeps the distance between the grounding point and the CoG close, an upward force can efficiently work for the STS assistance. To implement the strategy, this paper utilized the High-Strength and Flexible Mechanism (HSFM). The HSFM can change its grounding point while being capable of lifting body weight with one motor. Owing to these unique characteristics, the VGFL can realize the strategy without multiple actuators and complex controllers. Furthermore, real-world experiments confirmed that the grounding point of the VGFL accurately tracked the forward shift of the CoG during the STS assistance. Moreover, experiments conducted with three healthy subjects showed that the VGFL reduced the surface myoelectricity of the lower limbs during STS transfer. The VGFL could demonstrate high support performance in STS with the CoG tracking strategy.