Several barriers prevent the integration and adoption of augmented reality (AR) in robotic renal surgery despite the increased availability of virtual three-dimensional (3D) models. Apart from correct model alignment and deformation, not all instruments are clearly visible in AR. Superimposition of a 3D model on top of the surgical stream, including the instruments, can result in a potentially hazardous surgical situation. We demonstrate real-time instrument detection during AR-guided robot-assisted partial nephrectomy and show the generalization of our algorithm to AR-guided robot-assisted kidney transplantation. We developed an algorithm using deep learning networks to detect all nonorganic items. This algorithm learned to extract this information for 65 927 manually labeled instruments on 15 100 frames. Our setup, which runs on a standalone laptop, was deployed in three different hospitals and used by four different surgeons. Instrument detection is a simple and feasible way to enhance the safety of AR-guided surgery. Future investigations should strive to optimize efficient video processing to minimize the 0.5-s delay currently experienced. General AR applications also need further optimization, including detection and tracking of organ deformation, for full clinical implementation.

Introduction and Hypothesis: Robotic sacropexy (RSC) emerged in the last years as a valid alternative to the laparoscopic technique. However, the robotic approach is still limited by platform availability and concerns about cost-effectiveness. Recently, new robotic platforms joined the market, lowering the costs and offering the possibility to expand the robotic approach. The aim of our study was to demonstrate the technical feasibility and safety of the procedure with this new platform along with the description of our surgical setting. Materials and Methods: We reported data on the first five consecutive patients who underwent RSC at Onze Lieve Vrouw Hospital (Aalst, Belgium), performed with the novel HUGO™ Robot-Assisted Surgery (RAS) System. The platform consists of four fully independent carts, an open console, and a system tower equipped for both laparoscopic and robotic surgery. We collected patients' characteristics, intraoperative data, intraoperative complications, and clashes of instruments. Results: All procedures were completed according to the same surgical setting and technique. No need for conversion to open/laparoscopic surgery and/or for additional port placement was required. No intraoperative complications, instrument clashes, or system failure that compromised the surgery's completion were recorded. Median interquartile range docking, operative, and console time were 8 (6–9), 130 (115–165), and 80 (80–115) minutes, respectively. Conclusion: This series represents the first worldwide report of a robot-assisted sacropexy executed with the novel HUGO RAS System. Awaiting future investigation, this preliminary experience provides relevant data in terms of operative room settings and perioperative outcomes that might be helpful for future adopters of this platform.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre‐operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real‐time de‐occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors’ best knowledge, the first‐in‐human on edge deployment of a real‐time binary segmentation pipeline during three robot‐assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state‐of‐the‐art real‐time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real‐time binary segmentation of 37 non‐organic surgical items, which are never occluded during AR. The application features real‐time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.