In pathological diagnostics, histological images highlight the oncological features of excised specimens, but they require laborious and costly staining procedures. Despite recent innovations in label-free microscopy that simplify complex staining procedures, technical limitations and inadequate histological visualization are still problems in clinical settings. Here, we demonstrate an interconnected deep learning (DL)-based framework for performing automated virtual staining, segmentation, and classification in label-free photoacoustic histology (PAH) of human specimens. The framework comprises three components: (1) an explainable contrastive unpaired translation (E-CUT) method for virtual H&E (VHE) staining, (2) an U-net architecture for feature segmentation, and (3) a DL-based stepwise feature fusion method (StepFF) for classification. The framework demonstrates promising performance at each step of its application to human liver cancers. In virtual staining, the E-CUT preserves the morphological aspects of the cell nucleus and cytoplasm, making VHE images highly similar to real H&E ones. In segmentation, various features (e.g., the cell area, number of cells, and the distance between cell nuclei) have been successfully segmented in VHE images. Finally, by using deep feature vectors from PAH, VHE, and segmented images, StepFF has achieved a 98.00% classification accuracy, compared to the 94.80% accuracy of conventional PAH classification. In particular, StepFF’s classification reached a sensitivity of 100% based on the evaluation of three pathologists, demonstrating its applicability in real clinical settings. This series of DL methods for label-free PAH has great potential as a practical clinical strategy for digital pathology.

Mid-infrared photoacoustic microscopy can capture biochemical information without staining. However, the long mid-infrared optical wavelengths make the spatial resolution of photoacoustic microscopy significantly poorer than that of conventional confocal fluorescence microscopy. Here, we demonstrate an explainable deep learning-based unsupervised inter-domain transformation of low-resolution unlabeled mid-infrared photoacoustic microscopy images into confocal-like virtually fluorescence-stained high-resolution images. The explainable deep learning-based framework is proposed for this transformation, wherein an unsupervised generative adversarial network is primarily employed and then a saliency constraint is added for better explainability. We validate the performance of explainable deep learning-based mid-infrared photoacoustic microscopy by identifying cell nuclei and filamentous actins in cultured human cardiac fibroblasts and matching them with the corresponding CFM images. The XDL ensures similar saliency between the two domains, making the transformation process more stable and more reliable than existing networks. Our XDL-MIR-PAM enables label-free high-resolution duplexed cellular imaging, which can significantly benefit many research avenues in cell biology.

Although accurate detection of breast cancer still poses significant challenges, deep learning (DL) can support more accurate image interpretation. In this study, we develop a highly robust DL model based on combined B-mode ultrasound (B-mode) and strain elastography ultrasound (SE) images for classifying benign and malignant breast tumors. This study retrospectively included 85 patients, including 42 with benign lesions and 43 with malignancies, all confirmed by biopsy. Two deep neural network models, AlexNet and ResNet, were separately trained on combined 205 B-mode and 205 SE images (80% for training and 20% for validation) from 67 patients with benign and malignant lesions. These two models were then configured to work as an ensemble using both image-wise and layer-wise and tested on a dataset of 56 images from the remaining 18 patients. The ensemble model captures the diverse features present in the B-mode and SE images and also combines semantic features from AlexNet and ResNet models to classify the benign from the malignant tumors. The experimental results demonstrate that the accuracy of the proposed ensemble model is 90%, which is better than the individual models and the model trained using B-mode or SE images alone. Moreover, some patients that were misclassified by the traditional methods were correctly classified by the proposed ensemble method. The proposed ensemble DL model will enable radiologists to achieve superior detection efficiency owing to enhance classification accuracy for breast cancers in ultrasound (US) images.

Diffusion-weighted MRI is critical for diagnosing and managing ischemic stroke, but variability in images and disease presentation limits the generalizability of AI algorithms. We present DeepISLES, a robust ensemble algorithm developed from top submissions to the 2022 Ischemic Stroke Lesion Segmentation challenge we organized. By combining the strengths of best-performing methods from leading research groups, DeepISLES achieves superior accuracy in detecting and segmenting ischemic lesions, generalizing well across diverse axes. Validation on a large external dataset (N = 1685) confirms its robustness, outperforming previous state-of-the-art models by 7.4% in Dice score and 12.6% in F1 score. It also excels at extracting clinical biomarkers and correlates strongly with clinical stroke scores, closely matching expert performance. Neuroradiologists prefer DeepISLES’ segmentations over manual annotations in a Turing-like test. Our work demonstrates DeepISLES’ clinical relevance and highlights the value of biomedical challenges in developing real-world, generalizable AI tools. DeepISLES is freely available at https://github.com/ezequieldlrosa/DeepIsles.