Background: Addressing intraoperative bleeding remains a significant challenge in the field of robotic surgery. This research endeavors to pioneer a groundbreaking solution utilizing convolutional neural networks (CNNs). The objective is to establish a system capable of forecasting instances of intraoperative bleeding during robot-assisted radical prostatectomy (RARP) and promptly notify the surgeon about bleeding risks. Methods: To achieve this, a multi-task learning (MTL) CNN was introduced, leveraging a modified version of the U-Net architecture. The aim was to categorize video input as either “absence of blood accumulation” (0) or “presence of blood accumulation” (1). To facilitate seamless interaction with the neural networks, the Bleeding Artificial Intelligence-based Detector (BLAIR) software was created using the Python Keras API and built upon the PyQT framework. A subsequent clinical assessment of BLAIR’s efficacy was performed, comparing its bleeding identification performance against that of a urologist. Various perioperative variables were also gathered. For optimal MTL-CNN training parameterization, a multi-task loss function was adopted to enhance the accuracy of event detection by taking advantage of surgical tools’ semantic segmentation. Additionally, the Multiple Correspondence Analysis (MCA) approach was employed to assess software performance. Results: The MTL-CNN demonstrated a remarkable event recognition accuracy of 90.63%. When evaluating BLAIR’s predictive ability and its capacity to pre-warn surgeons of potential bleeding incidents, the density plot highlighted a striking similarity between BLAIR and human assessments. In fact, BLAIR exhibited a faster response. Notably, the MCA analysis revealed no discernible distinction between the software and human performance in accurately identifying instances of bleeding. Conclusion: The BLAIR software proved its competence by achieving over 90% accuracy in predicting bleeding events during RARP. This accomplishment underscores the potential of AI to assist surgeons during interventions. This study exemplifies the positive impact AI applications can have on surgical procedures.

In orthopedic surgery and maxillofacial there is a growing use of augmented reality (AR) as a technology to increase the visual perception of the surgeon in the operating room. The objective of this review is to analyze the state of the art in the use of AR for osteotomies, highlighting the advantages and the most-known open issues to be addressed in the future research. Scopus, Web of Science, Pubmed and IEEE Xplore databases have been explored with a keyword search, setting the time limits from January 2017 to January 2023, inclusive. Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines has been used in this review, focusing on anatomical districts, real-virtual environment interaction, advantaged and limitations of existing AR-based applications. 49 articles met the inclusion criteria and have been selected in the final analysis. For the sake of clarity, works have been grouped according to the anatomical district, but also the real-virtual environment interaction methodology was reported, as well as information regarding accuracy assessment. A Quality Function Deployment (QFD) has been used to assess the AR-based solutions with regards to the more traditional freehand (FH) and Patient Specific Template (PST) approaches. Finally, some suggestions to foster the AR-based solution adoption in osteotomies treatment have been drawn, considering the highlighted advantages and limitations of this technology. The AR resulted to meet the surgeons’ needs more than other traditional approaches. Among the emerged advantages, AR can lead to a better surgical field accessibility, more flexible solutions and lower the management effort. Nonetheless, future research should address some well-known issues, among which the calibration time, the robustness of the tracking, and the HMDs discomfort.

Cutting guides are widely used in cranio-maxillofacial surgery by providing mechanical references for precise bone resections and reducing intraoperative variability. Nevertheless, their rigid and patient-specific design requires dedicated CAD modeling and fabrication, making them time-consuming to produce and difficult to adapt when anatomical conditions or bone surfaces change. This work presents a hybrid surgical cutting guide that combines a physically adjustable device with Augmented Reality (AR) feedback to support intraoperative alignment in maxillofacial osteotomies. The concept merges the tactile reliability of conventional guides with the adaptability of digital visualization, enabling surgeons to fine-tune the cutting plane directly through AR tracking. Registration was performed using cephalometric landmarks and an inside-out tracking approach with HoloLens 2, allowing precise superimposition of virtual cutting planes onto 3D-printed mandibular models. The system was evaluated by both expert and novice operators under three feedback conditions: no AR, holographic overlay, and real-time distance guidance. Results showed that AR feedback was associated with improved positional accuracy, with mean linear deviations of 1.27±0.71mm and angular errors of 4.46±3.27°. Operator experience influenced overall performance, yet enhanced feedback compensated part of this variability. Combining physical and digital guidance can yield more adaptable, precise, and reusable osteotomy tools, paving the way for flexible surgical assistance in clinical settings.

Augmented Reality (AR) is increasingly being adopted in surgical procedures. Identifying accurate and reliable tracking systems can enhance the effectiveness of AR assisted surgery. The study presents a benchmarking platform to evaluate the performance of different optical surgical tools tracking systems for AR applications, addressing the need for standardized comparison of tracking systems’ accuracy, repeatability, and reliability. A custom-built, cost-effective benchmarking platform was developed, and different quantitative evaluation metrics were employed to assess performances of Marker-Based (MB) tracking systems. Two types of measurements were performed: static, where metrics such as Target Registration Error (TRE) and tip stability as a measure of jitter were estimated, and dynamic, involving a measure of the deviation between the tracked tooltip trajectory and its GT. Three AR MB-tracking methods were tested: method 1, mono-RGB sensor tracking a planar image target; method 2, RGB-depth sensor tracking passive spherical markers; and method 3, mono-IR sensor tracking IR active markers. Significant performance differences were observed. Method 3 achieved the lowest TRE value of $$3.578 \pm 2.836 \text { mm}.$$ Method 1 exhibited the best tip stability with a jitter value of $$1.435 \pm 0.824 \text { mm}.$$ Method 2 obtained the minimum trajectory distance in dynamic tests. The benchmarking platform demonstrated its effectiveness in evaluating and comparing different AR tracking methods. This study enables the optimization of tracking system selection for use in the operating room, and facilitating the integration of AR as a supportive technology in surgical practice.