Background: 3D reconstruction algorithms are of fundamental importance for augmented reality applications in computer‐assisted surgery. However, few datasets of endoscopic stereo images with associated 3D surface references are currently openly available, preventing the proper validation of such algorithms. This work presents a new and rich dataset of endoscopic stereo images (EndoAbS dataset).Methods: The dataset includes (i) endoscopic stereo images of phantom abdominal organs, (ii) a 3D organ surface reference (RF) generated with a laser scanner and (iii) camera calibration parameters. A detailed description of the generation of the phantom and the camera–laser calibration method is also provided.Results: An estimation of the overall error in creation of the dataset is reported (camera–laser calibration error 0.43 mm) and the performance of a 3D reconstruction algorithm is evaluated using EndoAbS, resulting in an accuracy error in accordance with state‐of‐the‐art results (<2 mm).Conclusions: The EndoAbS dataset contributes to an increase the number and variety of openly available datasets of surgical stereo images, including a highly accurate RF and different surgical conditions.

In abdominal surgery, intra-operative bleeding is one of the major complications that affect the outcome of minimally invasive surgical procedures. One of the causes is attributed to accidental damages to arteries or veins, and one of the possible risk factors falls on the surgeon's skills. This paper presents the development and application of an Enhanced Vision System for Robotic Surgery (EnViSoRS), based on a user-defined Safety Volume (SV) tracking to minimise the risk of intra-operative bleeding. It aims at enhancing the surgeon's capabilities by providing Augmented Reality (AR) assistance towards the protection of vessels from injury during the execution of surgical procedures with a robot. The core of the framework consists in: (i) a hybrid tracking algorithm (LT-SAT tracker) that robustly follows a user-defined Safety Area (SA) in long term; (ii) a dense soft tissue 3D reconstruction algorithm, necessary for the computation of the SV; (iii) AR features for visualisation of the SV to be protected and of a graphical gauge indicating the current distance between the instruments and the reconstructed surface. EnViSoRS was integrated with a commercial robotic surgery system (the dVRK system) for testing and validation. The experiments aimed at demonstrating the accuracy, robustness, performance and usability of EnViSoRS during the execution of a simulated surgical task on a liver phantom. Results show an overall accuracy in accordance with surgical requirements (< 5mm), and high robustness in the computation of the SV in terms of precision and recall of its identification. The optimisation strategy implemented to speed up the computational time is also described and evaluated, providing AR features update rate up to 4 fps without impacting the real-time visualisation of the stereo endoscopic video. Finally, qualitative results regarding the system usability indicate that the proposed system integrates well with the commercial surgical robot and has indeed potential to offer useful assistance during real surgeries.

Several studies have demonstrated that training with a laparoscopic simulator improves laparoscopic technical skills. We describe how to build a homemade, low-cost laparoscopic training simulator (LABOT) and its validation as a training instrument. First, sixty surgeons filled out a survey characterized by 12 closed-answer questions about realism, ergonomics, and usefulness for surgical training (global scores ranged from 1—very insufficient to 5—very good). The results of the questionnaires showed a mean (±SD) rating score of 4.18 ± 0.65 for all users. Then, 15 students (group S) and 15 residents (group R) completed 3 different tasks (T1, T2, T3), which were repeated twice to evaluate the execution time and the number of users’ procedural errors. For T1, the R group had a lower mean execution time and a lower rate of procedural errors than the S group; for T2, the R and S groups had a similar mean execution time, but the R group had a lower rate of errors; and for T3, the R and S groups had a similar mean execution time and rate of errors. On a second attempt, all the participants tended to improve their results in doing these surgical tasks; nevertheless, after subgroup analysis of the T1 results, the S group had a better improvement of both parameters. Our laparoscopic simulator is simple to build, low-cost, easy to use, and seems to be a suitable resource for improving laparoscopic skills. In the future, further studies should evaluate the potential of this laparoscopic box on long-term surgical training with more complex tasks and simulation attempts.

Augmented Reality (AR) has gained significant interest within the research community in the past two decades. In surgery, AR overlays critical information directly onto the surgeon's visual field, thus enhancing situational awareness by providing navigation guidance and contributing to safer, more precise, and more efficient surgical interventions. This review examines registration methods suitable for laparoscopic scenarios involving a pre-operative 3D model and an intra-operative 2D or 3D video, both in rigid and non-rigid conditions. We started analysing traditional methods, which do not include Deep Learning (DL). However, in recent years, DL has revolutionized the landscape of computer vision in many tasks. So, we investigated these methods in our scope, identifying two main categories: hybrid DL-enhanced methods and DL point cloud registration methods. The former applies DL across different stages of the traditional surface-based methods to enhance their performances. The latter uses end-to-end approaches to estimate the transformation matrix between two input point clouds. For each category discussed, we highlight both the strengths and weaknesses associated with the challenges of surgical AR to aid comprehension, even for those less familiar with the topic.