Large Language Models (LLMs) are gaining popularity in the field of robotics. However, LLM-based robots are limited to simple, repetitive motions due to the poor integration between language models, robots, and the environment. This letter proposes a novel approach to enhance the performance of LLM-based autonomous manipulation through Human-Robot Collaboration (HRC). The approach involves using a prompted GPT-4 language model to decompose high-level language commands into sequences of motions that can be executed by the robot. The system also employs a YOLO-based perception algorithm, providing visual cues to the LLM, which aids in planning feasible motions within the specific environment. Additionally, an HRC method is proposed by combining teleoperation and Dynamic Movement Primitives (DMP), allowing the LLM-based robot to learn from human guidance. Real-world experiments have been conducted using the Toyota Human Support Robot for manipulation tasks. The outcomes indicate that tasks requiring complex trajectory planning and reasoning over environments can be efficiently accomplished through the incorporation of human demonstrations.

Haptic feedback systems play a critical role in enriching the user experience in human-robot interaction. However, existing devices designed for evoking haptic sensations often face limitations owing to their low degree of freedom of deformation. In this study, we introduce the Funabot-Sleeve, a haptic device based on McKibben artificial muscles, and investigate its potential to evoke a range of haptic sensations using both steady-state and transient air pressure patterns. Our investigation examines the influence of these patterns on evoking distinct haptic sensations and identifies four specific sensations that can be evoked: Embraced, Pinched, a combination of Embraced and Pressed, and Twisted sensations. Across all participants, the evoked sensations showed positive correlations, with most correlations exceeding a value of 0.4, indicating a high degree of agreement in the sensations felt by the subjects. Our research lays the groundwork for the design of fabric actuators, capable of replicating specific stimuli and skin surface effects, thereby enabling a more sophisticated and personalized haptic feedback experience.

This study proposes a concept for multithread active vision sensing that can measure dynamically changing displacement and vibration at multiple points on civil engineering structures. In multithread active vision sensing, a high-speed camera can function virtually as multiple tracking cameras by accelerating its measurement, computation, and actuation with ultrafast viewpoint switching at millisecond level. This enables simultaneous measurement of small vibrations distributed across a wide range, which cannot be observed using a single camera system, because its pixel accuracy is generally incompatible with its wide angle of view. We developed a galvanomirror-based high-speed multithread active vision system that can switch 500 different views in a second; it functioned as 15 virtual cameras each operating at 33.3 fps to observe multiple scenes in completely different views. The experimental results for a 4-m-long truss-structure bridge model to which 15 markers were attached show that a single active vision system can observe the deformation of the bridge structure and estimate modal parameters, such as resonant frequencies and mode shapes, at a frequency on the order of dozens of hertz.

This paper presents our development of an intuitive teleoperation system for Toyota's Human Support Robot. While this robot has been widely used as an autonomous agent in the field of service robotics, we explore a possibility of this robot for the use of remote assistance of daily activities in home environments. For rapid prototyping an affordable teleoperation system, we adopt a commercially available VR device as a user interface and develop a Robot Operating System (ROS)-based control framework building on the software application programming interfaces (APIs). In particular, we implement a singularity robust inverse kinematics algorithm to achieve a coordinated whole-body motion of the robot for safe and intuitive operation. We empirically evaluate the practical feasibility of the proposed teleoperation system through various test tasks. We discuss the potential and limitations of the current development for its future improvement.

The reduced workspace in endonasal endoscopic surgery (EES) hinders the execution of complex surgical tasks such as suturing. Typically, surgeons need to manipulate non-dexterous long surgical instruments with an endoscopic view that makes it difficult to estimate the distances and angles required for precise suturing motion. Recently, robot-assisted surgical systems have been used in laparoscopic surgery with promising results. Although robotic systems can provide enhanced dexterity, robot-assisted suturing is still highly challenging. In this paper, we propose a robot-assisted stitching method based on an online optimization-based trajectory generation for curved needle stitching and a constrained motion planning framework to ensure safe surgical instrument motion. The needle trajectory is generated online by using a sequential convex optimization algorithm subject to stitching kinematic constraints. The constrained motion planner is designed to reduce surrounding damages to the nasal cavity by setting a remote center of motion over the nostril. A dual concurrent inverse kinematics (IK) solver is proposed to achieve convergence of the solution and optimal time execution, in which two constrained IK methods are performed simultaneously; a task-priority based IK and a nonlinear optimization-based IK. We evaluate the performance of the proposed method in a stitching experiment with our surgical robotic system in a robot-assisted mode and an autonomous mode in comparison to the use of a conventional surgical tool. Our results demonstrate a noticeable improvement in the stitching success ratio in the robot-assisted mode and the shortest completion time for the autonomous mode. In addition, the force interaction with the tissue was highly reduced when using the robotic system.