6 August 2025 in Computerized Medical Imaging and Graphics
Augmented Reality (AR) has gained significant interest within the research community in the past two decades. In surgery, AR overlays critical information directly onto the surgeon's visual field, thus enhancing situational awareness by providing navigation guidance and contributing to safer, more precise, and more efficient surgical interventions. This review examines registration methods suitable for laparoscopic scenarios involving a pre-operative 3D model and an intra-operative 2D or 3D video, both in rigid and non-rigid conditions. We started analysing traditional methods, which do not include Deep Learning (DL). However, in recent years, DL has revolutionized the landscape of computer vision in many tasks. So, we investigated these methods in our scope, identifying two main categories: hybrid DL-enhanced methods and DL point cloud registration methods. The former applies DL across different stages of the traditional surface-based methods to enhance their performances. The latter uses end-to-end approaches to estimate the transformation matrix between two input point clouds. For each category discussed, we highlight both the strengths and weaknesses associated with the challenges of surgical AR to aid comprehension, even for those less familiar with the topic.
30 July 2024 in Frontiers in Digital Health
Introduction: The use of robotic systems in the surgical domain has become groundbreaking for patients and surgeons in the last decades. While the annual number of robotic surgical procedures continues to increase rapidly, it is essential to provide the surgeon with innovative training courses along with the standard specialization path. To this end, simulators play a fundamental role. Currently, the high cost of the leading VR simulators limits their accessibility to educational institutions. The challenge lies in balancing high-fidelity simulation with cost-effectiveness; however, few cost-effective options exist for robotic surgery training.Methods: This paper proposes the design, development and user-centered usability study of an affordable user interface to control a surgical robot simulator. It consists of a cart equipped with two haptic interfaces, a VR visor and two pedals. The simulations were created using Unity, which offers versatility for expanding the simulator to more complex scenes. An intuitive teleoperation control of the simulated robotic instruments is achieved through a high-level control strategy.Results and Discussion: Its affordability and resemblance to real surgeon consoles make it ideal for implementing robotic surgery training programs in medical schools, enhancing accessibility to a broader audience. This is demonstrated by the results of an usability study involving expert surgeons who use surgical robots regularly, expert surgeons without robotic surgery experience, and a control group. The results of the study, which was based on a traditional Peg-board exercise and Camera Control task, demonstrate the simulator’s high usability and intuitive control across diverse user groups, including those with limited experience. This offers evidence that this affordable system is a promising solution for expanding robotic surgery training.
17 November 2025 in Frontiers in Robotics and AI
Objective: Registering a preoperative 3D model of an organ with its actual anatomy viewed from an intraoperative video is a fundamental challenge in computer-assisted surgery, especially for surgical augmented reality. To address this, we present a benchmark of state-of-the-art deep learning point-cloud registration methods, offering a transparent evaluation of their generalizability to surgical scenarios and establishing a robust guideline for developing advanced non-rigid algorithms. Methods: We systematically evaluate traditional and deep learning GMM-based, correspondence-based, correspondence-free, matching-based, and liver-specific point cloud registration approaches on two surgical datasets: a deformed IRCAD liver set and DePoll dataset. We also propose our complete-to-partial point cloud registration framework that leverages keypoint extraction, overlap estimation, and a Transformer-based architecture, culminating in competitive registration results. Results: Experimental evaluations on deformed IRCAD tests reveal that most deep learning methods achieve good registration performances with TRE<10 mm, MAE(R) < 4 and MAE(t)<5 mm. On DePoll, however, performance drops dramatically due to the large deformations. Conclusion: In conclusion, deep-learning rigid registration methods remain reliable under small deformations and varying partiality but lose accuracy when faced with severe non-rigid changes. To overcome this, future work should focus on building non-rigid registration architectures that preserve the strengths of self-, cross-attention and overlap modules while enhancing correspondence estimation to handle large deformations in laparoscopic surgery.
19 June 2025 in International Journal of Computer Assisted Radiology and Surgery