Background: Integrating simulators with robotic surgical procedures could assist in designing and testing of novel robotic control algorithms and further enhance patient‐specific pre‐operative planning and training for robotic surgeries.Methods: A virtual reality simulator, developed to perform the transsphenoidal resection of pituitary gland tumours, tested the usability of robotic interfaces and control algorithms. It used position‐based dynamics to allow soft‐tissue deformation and resection with haptic feedback; dynamic motion scaling control was also incorporated into the simulator.Results: Neurosurgeons and residents performed the surgery under constant and dynamic motion scaling conditions (CMS vs DMS). DMS increased dexterity and reduced the risk of damage to healthy brain tissue. Post‐experimental questionnaires indicated that the system was well‐evaluated by experts.Conclusion: The simulator was intuitively and realistically operated. It increased the safety and accuracy of the procedure without affecting intervention time. Future research can investigate incorporating this simulation into a real micro‐surgical robotic system.

The value of kinematic data for skill assessment is being investigated. This is the first virtual reality simulator developed for liver surgery. This simulator was coded in C++ using PhysX and FleX with a novel cutting algorithm and used a patient data-derived model and two instruments functioning as ultrasonic shears. The simulator was evaluated by nine expert surgeons and nine surgical novices. Each participant performed a simulated metastasectomy after training. Kinematic data were collected for the instrument position. Each participant completed a survey. The expert participants had a mean age of 47 years and 9/9 were certified in surgery. Novices had a mean age of 30 years and 0/9 were certified surgeons. The mean path length (novice 0.76 ± 0.20 m vs. expert 0.46 ± 0.16 m, p = 0.008), movements (138 ± 45 vs. 84 ± 32, p = 0.043) and time (174 ± 44 s vs. 102 ± 42 s, p = 0.004) were significantly different for the two participant groups. There were no significant differences in activating the instrument (107 ± 25 vs. 109 ± 53). Participants considered the simulator realistic (6.5/7) (face validity), appropriate for education (5/7) (content validity) with an effective interface (6/7), consistent motion (5/7) and realistic soft tissue behavior (5/7). This study showed that the simulator differentiates between experts and novices. Simulation may be an effective way to obtain kinematic data.

Purpose: To support research on autonomous robotic micro-drilling for cranial window creation in mice, a multimodal digital twin (DT) is developed to generate realistic synthetic images and drilling sounds. The realism of the DT is evaluated using data from an eggshell drilling scenario, demonstrating its potential for training AI models with multimodal synthetic data. Methods: The asynchronous multi-body framework (AMBF) simulator for volumetric drilling with haptic feedback is combined with the Isaac Sim simulator for photorealistic rendering. A deep audio generator (DAG) model is presented and its realism is evaluated on real drilling sounds. A convolutional neural network (CNN) trained on synthetic images is used to assess visual realism by detecting drilling areas in real eggshell images. Finally, the accuracy of the DT is evaluated by experiments on a real eggshell. Results: The DAG model outperformed pitch modulation methods, achieving lower Frechet audio distance (FAD) and Frechet inception distance (FID) scores, demonstrating a closer resemblance to real drilling sounds. The CNN trained on synthetic images achieved a mean average precision (mAP) of 70.2 when tested on real drilling images. The DT had an alignment error of 0.22 ± 0.03 mm. Conclusion: A multimodal DT has been developed to simulate the creation of the cranial window on an eggshell model and its realism has been evaluated. The results indicate a high degree of realism in both the synthetic audio and images and submillimeter accuracy.