We developed a fully automated, two-step deep learning approach for characterizing coronary calcified plaque in intravascular optical coherence tomography (IVOCT) images. First, major calcification lesions were detected from an entire pullback using a 3D convolutional neural network (CNN). Second, a SegNet deep learning model with the Tversky loss function was used to segment calcified plaques in the major calcification lesions. The fully connected conditional random field and the frame interpolation of the missing calcification frames were used to reduce classification errors. We trained/tested the networks on a large dataset comprising 8,231 clinical images from 68 patients with 68 vessels and 4,320 ex vivo cadaveric images from 4 hearts with 4 vessels. The 3D CNN model detected major calcifications with high sensitivity (97.7%), specificity (87.7%), and F1 score (0.922). Compared to the standard one-step approach, our two-step deep learning approach significantly improved sensitivity (from 77.5% to 86.2%), precision (from 73.5% to 75.8%), and F1 score (from 0.749 to 0.781). We investigated segmentation performance for varying numbers of training samples; at least 3,900 images were required to obtain stable segmentation results. We also found very small differences in calcification attributes (e.g., angle, thickness, and depth) and identical calcium scores on repetitive pullbacks, indicating excellent reproducibility. Applied to new clinical pullbacks, our method has implications for real-time treatment planning and imaging research.

Major calcifications are of great concern when performing percutaneous coronary intervention (PCI) because they can hinder stent deployment. Approximately 700,000 PCIs are performed each year, and many involve the use of stents to open up obstructed coronary arteries.¹ Calcified plaques are found in 17% to 35% of patients undergoing PCI.^{2 – 4} Calcifications can lead to stent underexpansion and strut malapposition, which in turn can lead to increased risk of thrombosis and in-stent restenosis.^{5 – 10} A cardiologist has several options when confronting a calcified lesion: high balloon pressures (up to 30 atm) to fracture the calcification, scoring balloon, Shockwave^TM intravascular lithotripsy (IVL), rotational atherectomy, etc. In some cases, the lesion may not be treatable.

Deep learning based methods are routinely used to segment various structures of interest in varied medical imaging modalities. Acquiring annotations for a large number of images requires a skilled analyst, and the process is both time consuming and challenging. Our approach to reduce effort is to reduce the number of images needing detailed annotation. For intravascular optical coherence tomography (IVOCT) image pullbacks, we tested 10% to 100% of training images derived from two schemes: equally-spaced image subsampling and deep-learning- based image clustering. The first strategy involves selecting images at equally spaced intervals from the volume, accounting for the high spatial correlation between neighboring images. In clustering, we used an autoencoder to create a deep feature space representation, performed k-medoids clustering, and then used the cluster medians for training. For coronary calcifications, a baseline U-net model was trained on all images from volumes of interest (VOIs) and compared with fewer images from the sub-sampling strategies. For a given sampling ratio, the clustering based method performed better or similar as compared to the equally spaced sampling approach (e.g., with 10% of images, mean F1 score for calcific class increased from 0.52 to 0.63, with equal spacing and with clustering, respectively). Additionally, for a fixed number of training images, sampling images from more VOIs performed better than otherwise. In conclusion, we recommend the clustering based approach to annotate a small fraction of images, creating a baseline model, which potentially can be improved further by annotating images selected using methods described in active learning research.

It can be difficult/impossible to fully expand a coronary artery stent in a heavily calcified coronary artery lesion. Under-expanded stents are linked to later complications. Here we used machine/deep learning to analyze calcifications in pre-stent intravascular optical coherence tomography (IVOCT) images and predicted the success of vessel expansion. Pre- and post-stent IVOCT image data were obtained from 110 coronary lesions. Lumen and calcifications in pre-stent images were segmented using deep learning, and lesion features were extracted. We analyzed stent expansion along the lesion, enabling frame, segmental, and whole-lesion analyses. We trained regression models to predict the post-stent lumen area and then computed the stent expansion index (SEI). Best performance (root-mean-square-error = 0.04 ± 0.02 mm²,r = 0.94 ± 0.04,p < 0.0001) was achieved when we used features from both lumen and calcification to train a Gaussian regression model for segmental analysis of 31 frames in length. Stents with minimum SEI > 80% were classified as “well-expanded;” others were “under-expanded.” Under-expansion classification results (e.g., AUC = 0.85 ± 0.02) were significantly improved over a previous, simple calculation, as well as other machine learning solutions. Promising results suggest that such methods can identify lesions at risk of under-expansion that would be candidates for intervention lesion preparation (e.g., atherectomy).

Compared with other imaging modalities, intravascular optical coherence tomography (IVOCT) has significant advantages for guiding percutaneous coronary interventions, assessing their outcomes, and characterizing plaque components. To aid IVOCT research studies, we developed the Optical Coherence TOmography PlaqUe and Stent (OCTOPUS) analysis software, which provides highly automated, comprehensive analysis of coronary plaques and stents in IVOCT images. User specifications for OCTOPUS were obtained from detailed, iterative discussions with IVOCT analysts in the Cardiovascular Imaging Core Laboratory at University Hospitals Cleveland Medical Center, a leading laboratory for IVOCT image analysis. To automate image analysis results, the software includes several important algorithmic steps: pre-processing, deep learning plaque segmentation, machine learning identification of stent struts, and registration of pullbacks for sequential comparisons. Intuitive, interactive visualization and manual editing of segmentations were included in the software. Quantifications include stent deployment characteristics (e.g., stent area and stent strut malapposition), strut level analysis, calcium angle, and calcium thickness measurements. Interactive visualizations include (x,y) anatomical, en face, and longitudinal views with optional overlays (e.g., segmented calcifications). To compare images over time, linked visualizations were enabled to display up to four registered vessel segments at a time. OCTOPUS has been deployed for nearly 1 year and is currently being used in multiple IVOCT studies. Underlying plaque segmentation algorithm yielded excellent pixel-wise results (86.2% sensitivity and 0.781 F1 score). Using OCTOPUS on 34 new pullbacks, we determined that following automated segmentation, only 13% and 23% of frames needed any manual touch up for detailed lumen and calcification labeling, respectively. Only up to 3.8% of plaque pixels were modified, leading to an average editing time of only 7.5 s/frame, an approximately 80% reduction compared to manual analysis. Regarding stent analysis, sensitivity and precision were both greater than 90%, and each strut was successfully classified as either covered or uncovered with high sensitivity (94%) and specificity (90%). We demonstrated use cases for sequential analysis. To analyze plaque progression, we loaded multiple pullbacks acquired at different points (e.g., pre-stent, 3-month follow-up, and 18-month follow-up) and evaluated frame-level development of in-stent neo-atherosclerosis. In ex vivo cadaver experiments, the OCTOPUS software enabled visualization and quantitative evaluation of irregular stent deployment in the presence of calcifications identified in pre-stent images. We introduced and evaluated the clinical application of a highly automated software package, OCTOPUS, for quantitative plaque and stent analysis in IVOCT images. The software is currently used as an offline tool for research purposes; however, the software's embedded algorithms may also be useful for real-time treatment planning.