Purpose: Understanding the properties and aspects of the robotic system is essential to a successful medical intervention, as different capabilities and limits characterize each. Robot positioning is a crucial step in the surgical setup that ensures proper reachability to the desired port locations and facilitates docking procedures. This very demanding task requires much experience to master, especially with multiple trocars, increasing the barrier of entry for surgeons in training. Methods: Previously, we demonstrated an Augmented Reality-based system to visualize the rotational workspace of the robotic system and proved it helps the surgical staff to optimize patient positioning for single-port interventions. In this work, we implemented a new algorithm to allow for an automatic, real-time robotic arm positioning for multiple ports. Results: Our system, based on the rotational workspace data of the robotic arm and the set of trocar locations, can calculate the optimal position of the robotic arm in milliseconds for the positional and in seconds for the rotational workspace in virtual and augmented reality setups. Conclusions: Following the previous work, we extended our system to support multiple ports to cover a broader range of surgical procedures and introduced the automatic positioning component. Our solution can decrease the surgical setup time and eliminate the need to repositioning the robot mid-procedure and is suitable both for the preoperative planning step using VR and in the operating room—running on an AR headset.

When using redundant medical robots for hand-guiding heavy endoscopic tools, admittance control allows a completely defined tool and robot null space motion control. Although not a novel concept, comparative studies that help to design and implement admittance control are missing. In a user study, we compared four admittance controllers: one zero-torque controller that used measured joint torques and three others differently mapping forces/torques measured at the handle to tool tip velocity. We found that two of the four controllers (Dynamic MassandIntegrated Mass) outperformed the others.

We are developing a robotic system for future application in minimally invasive laser osteotomy. This paper presents the mechanical system concept as a macro-milli-micro system and focuses on designing and evaluating the milli-system. The milli-system consists of an articulated tendon-driven robotic endoscope with seven rigid links with an outer diameter of 8mm connected by six discrete rotational joints (±30°). These joints can be controlled individually, however, controlling one joint’s motion influences all joints located more distally, making joint control an interesting challenge. Controlling each joint as desired will allow positioning the micro-system mounted at the endoscope’s tip. The micro-system is itself a robot that will accurately position the laser. The robotic endoscope incorporates a hollow core with a diameter of 4.8mm that holds a supply channel for the micro-system with the necessary means for actuation and surgical intervention. We demonstrated the functionality of the robotic endoscope in tracking experiments. Despite the joints’ mutual influence, the articulated robotic endoscope could be handled successfully and achieved an angular settling error of less than 1° in the individual joints. The overall robotic system’s functionality was successfully demonstrated with a time-synchronized joint movement of the macro-system (serial manipulator) and the robotic endoscope.

Medical robots should not collide with close by obstacles during medical procedures, such as lamps, screens, or medical personnel. Redundant robots have more degrees of freedom than needed for moving endoscopic tools during surgery and can be reshaped to avoid obstacles by moving purely in the space of these additional degrees of freedom (null space). Although state-of-the-art robots allow surgeons to hand-guide endoscopic tools, reshaping the robot in null space is not intuitive for surgeons. Here we propose a learned task space control that allows surgeons to intuitively teach preferred robot configurations (shapes) that avoid obstacles using a VR-based planner in simulation. Later during surgery, surgeons control both the endoscopic tool and robot configuration (shape) with one hand. In a user study, we found that learned task space control outperformed state-of-the-art naive task space control in all the measured performance metrics (time, effort, and user-perceived effort). Our solution allowed users to intuitively interact with robots in VR and reshape robots while moving tools in medical and industrial applications.

In robot-assisted surgeries, the surgeon focuses on the surgical tool and its pose, and not on the complete robot’s shape. However, the joints of redundant robots (robots that have more degrees of freedom (DoF) than needed for the positioning of surgical tools) might move in unexpected/undesired ways. Joint motions that lead to patient or collisions are safety critical. We assume that the medical personnel in the operating room can best decide if a planned robot motions come too close to the patient or not. Therefore, we propose an augmented reality-based solution to interact with the robot during surgery planning, and intervention. The tool can be used to command a robot by drawing a trajectory in augmented reality (AR), visualizing the robot movement to check if it is safe before execution. The proposed solution allows surgeons to plan safe robot motion paths before-hand and adapt them when necessary in situ. As a proof-of-concept, we implemented and demonstrated the proposed solution on a 7-DoF redundant robot by commanding different trajectories. The control architecture to plan and execute motion for a surgical robot using AR is a key result of this work.