Large language models (LLMs) have gained increasing popularity in robotic task planning due to their exceptional abilities in text analytics and generation, as well as their broad knowledge of the world. However, they fall short in decoding visual cues. LLMs have limited direct perception of the world, which leads to a deficient grasp of the current state of the world. By contrast, the emergence of visual language models (VLMs) fills this gap by integrating visual perception modules, which can enhance the autonomy of robotic task planning. Despite these advancements, VLMs still face challenges, such as the potential for task execution errors, even when provided with accurate instructions. To address such issues, this letter proposes a ReplanVLM framework for robotic task planning. In this study, we focus on error correction interventions. An internal error correction mechanism and an external error correction mechanism are presented to correct errors under corresponding phases. A replan strategy is developed to replan tasks or correct error codes when task execution fails. Experimental results on real robots and in simulation environments have demonstrated the superiority of the proposed framework, with higher success rates and robust error correction capabilities in open-world tasks.

Minimally invasive procedures have gained ever-increasing popularity due to their advantages of smaller incisions, faster recoveries, fewer complications, and reduced scarring to name a few. With the force exertion and curvilinear flexibility at their distal end effectors, the continuum tubular robots have the potential to perform robot-assisted trans-orifice minimally invasive surgery, such as transnasal and transoral operations. During these procedures, it is important and challenging for the continuum tubular robot to automatically adjust its pose according to the target surgical sites and compensate for undesired disturbance, such as respiratory motions. In this paper, a singularity-avoidance visual servoing algorithm based on Jacobian Optimization has been proposed to improve safety and control continuum tubular robots based on intra-operative visual feedback in confined environments. Without the prior knowledge of robot kinematics or hand-eye calibration between the robot and the endoscope, the proposed model-free eye-in-hand visual servoing technique is capable of accomplishing the targeting tasks by adopting the safety-enhanced singularity avoidance mechanism and an efficient image-processing algorithm. The multiple experiments, including simulations, experiments conducted in a confined environment, and cadaveric studies, have been implemented and demonstrated to illustrate the superiority of the proposed singularity-avoidance visual servoing method.