In endovascular and cardiovascular surgery, real-time guidewire morphological and positional analysis is an important pre-requisite for robot-assisted intervention, which can aid in reducing radiation dose, contrast agent and procedure time. Nevertheless, this task often comes with the challenge of the deformable elongated structure with low contrast in noisy X-ray fluoroscopy. In this paper, a real-time multi-functional framework is proposed for fully automatic guidewire morphological and positional analysis, namely guidewire segmentation, endpoint localization and angle measurement. In the first stage, the proposed Fast Attention Recurrent Network (FAR-Net) achieves real-time and accurate guidewire segmentation. In the second stage, the endpoint localization and angle measurement algorithm robustly obtain sub-pixel-level endpoint and angle of the guidewire tip. Quantitative and qualitative evaluations on MSGSeg dataset consisting of 180 X-ray sequences from 30 patients demonstrate that the proposed framework significantly outperforms simpler baselines as well as the best previously-published result for this task. The proposed approach reached F1-Score of 0.938, mean distance error of 0.596 pixels, endpoint localization & angle measurement accuracy of 97.8% & 95.3%, and inference rate of approximately 13 FPS. The proposed framework not only addresses the issues of extreme class imbalance and misclassified examples, but also meets the real-time requirements, achieving the state-of-the-art performance. The proposed approach is promising for integration into robotic navigation frameworks to various intravascular applications, enabling robotic-assisted intervention.

The real-time localization of the guidewire endpoints is a stepping stone to computer-assisted percutaneous coronary intervention (PCI). However, methods for multi-guidewire endpoint localization in fluoroscopy images are still scarce. In this paper, we introduce a framework for real-time multi-guidewire endpoint localization in fluoroscopy images. The framework consists of two stages, first detecting all guidewire instances in the fluoroscopy image, and then locating the endpoints of each single guidewire instance. In the first stage, a YOLOv3 detector is used for guidewire detection, and a post-processing algorithm is proposed to refine the guidewire detection results. In the second stage, a Segmentation Attention-hourglass (SA-hourglass) network is proposed to predict the endpoint locations of each single guidewire instance. The SA-hourglass network can be generalized to the keypoint localization of other surgical instruments. In our experiments, the SA-hourglass network is applied not only on a guidewire dataset but also on a retinal microsurgery dataset, reaching the mean pixel error (MPE) of 2.20 pixels on the guidewire dataset and the MPE of 5.30 pixels on the retinal microsurgery dataset, both achieving the state-of-the-art localization results. Besides, the inference rate of our framework is at least 20FPS, which meets the real-time requirement of fluoroscopy images (6-12FPS).

Objective: Technical skills assessment plays an important role in the professional development of an interventionalist in percutaneous coronary intervention (PCI), however, most of traditional assessment methods are time-consuming and subjective. This paper aims to develop objective assessment techniques. Methods: In this study, a natural-behavior-based analysis framework is proposed to qualitatively and quantitatively assess technical skills in PCI. In vivo porcine studies were conducted to deliver a medical guidewire to two target coronaries of left circumflex arteries by six novice and four expert interventionalists. Simultaneously, four types of natural behaviors (i.e., hand motion, proximal force, muscle activity and finger motion) were acquired from the subjects' dominant hand and arm. The features extracted from behaviors of different skill-level groups were compared using Mann-Whitney U-test for effective behavior selection. The effective ones were further applied in Gaussian-mixture-model-based qualitative assessment and Mahalanobis-distance-based quantitative assessment. Results: The qualitative assessment achieves 92% accuracy to distinguish the novice and expert attempts, which is significantly higher than that of using single guidewire motions. Furthermore, the quantitative assessment can assign objective and effective scores for all attempts, indicating high correlation (R=0.9225) to those obtained by traditional methods. Conclusion: The objective, effective and comprehensive assessment of technical skills can be provided by qualitatively and quantitatively analyzing interventionalists' natural behaviors in PCI. Significance: This paper suggests a novel approach for technical skills assessment and the promising results demonstrate the great importance and effectiveness of the proposed method for promoting the development of objective assessment techniques.

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners —This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.