Tracking the precise movement of surgical tools is essential for enabling automated analysis, providing feedback, and enhancing safety in robotic-assisted surgery. Accurate 3D tracking of surgical tooltips is challenging to implement when using monocular videos due to the complexity of extracting depth information. We propose a pipeline that combines state-of-the-art foundation models—Florence2 and Segment Anything 2 (SAM2)—for zero-shot 2D localization of tooltip coordinates using a monocular video input. Localization predictions are refined through supervised training of the YOLOv11 segmentation model to enable real-time applications. The depth estimation model Metric3D computes the relative depth and provides tooltip camera coordinates, which are subsequently transformed into world coordinates via a linear model estimating rotation and translation parameters. An experimental evaluation on the JIGSAWS Suturing Kinematic dataset achieves a 3D Average Jaccard score on tooltip tracking of 84.5 and 91.2 for the zero-shot and supervised approaches, respectively. The results validate the effectiveness of our approach and its potential to enhance real-time guidance and assessment in robotic-assisted surgical procedures.

Objective: To determine whether multimodal predictive models incorporating task-level objective performance indicators (OPIs) alongside preoperative clinical variables improve prediction of surgeon experience, case complexity, and outcomes in robotic cholecystectomy. Background: OPIs from robotic platforms enable automated assessment of intraoperative performance, but prior multimodal analyses have suggested limited incremental value beyond clinical variables. Whether this conclusion holds when OPIs are computed at task level and target specifically curated endpoints remains untested. Methods: We retrospectively analyzed 234 Calot’s triangle dissections, 198 by attending surgeons and 36 by trainees. Three feature sets per target—clinical-only (10 preoperative variables), OPI-only (10 target-specific kinematic features), and combined—were evaluated using nested 5-seed×5-fold cross-validation across 5 algorithms (logistic regression, support vector machine, random forest, XGBoost, LightGBM). Discriminative performance was compared by paired Wilcoxon signed-rank tests on fold-level area under the receiver operating characteristic curve (AUROC). Feature contributions were quantified with SHapley Additive exPlanations (SHAP) analysis. Results: For surgeon experience and case complexity, OPIs alone matched the combined model (AUROC: 0.967 vs. 0.964; 0.791 vs. 0.792; both P >0.05) and outperformed clinical variables alone ( P <0.001 and P =0.006). For outcomes, the combined model significantly exceeded both single-modality models (AUROC: 0.753 vs. 0.720 clinical and 0.720 OPI-only; P =0.015 and 0.005). OPIs contributed 90.7%, 73.3%, and 51.7% of total absolute SHAP value across the 3 targets, decreasing with distance from intraoperative kinematics. Conclusions: Task-level, target-specific OPIs carry independent discriminative information beyond routine preoperative clinical variables. Their incremental value is target-dependent, with multimodal integration offering statistically significant but modest gains for outcomes.

We introduce specialized diffusion-based generative models that capture the spatiotemporal dynamics of fine-grained robotic surgical sub-stitch actions through supervised learning on annotated laparoscopic surgery footage. The proposed models form a foundation for data-driven world models capable of simulating the biomechanical interactions and procedural dynamics of surgical suturing with high temporal fidelity. Annotating a dataset of $\sim2K$ clips extracted from simulation videos, we categorize surgical actions into fine-grained sub-stitch classes including ideal and non-ideal executions of needle positioning, targeting, driving, and withdrawal. We fine-tune two state-of-the-art video diffusion models, LTX-Video and HunyuanVideo, to generate high-fidelity surgical action sequences at $\ge$768x512 resolution and $\ge$49 frames. For training our models, we explore both Low-Rank Adaptation (LoRA) and full-model fine-tuning approaches. Our experimental results demonstrate that these world models can effectively capture the dynamics of suturing, potentially enabling improved training simulators, surgical skill assessment tools, and autonomous surgical systems. The models also display the capability to differentiate between ideal and non-ideal technique execution, providing a foundation for building surgical training and evaluation systems. We release our models for testing and as a foundation for future research. Project Page: https://mkturkcan.github.io/suturingmodels/