Deep learning algorithms have been successfully used in medical image classification. In the next stage, the technology of acquiring explainable knowledge from medical images is highly desired. Here we show that deep learning algorithm enables automated acquisition of explainable features from diagnostic annotation-free histopathology images. We compare the prediction accuracy of prostate cancer recurrence using our algorithm-generated features with that of diagnosis by expert pathologists using established criteria on 13,188 whole-mount pathology images consisting of over 86 billion image patches. Our method not only reveals findings established by humans but also features that have not been recognized, showing higher accuracy than human in prognostic prediction. Combining both our algorithm-generated features and human-established criteria predicts the recurrence more accurately than using either method alone. We confirm robustness of our method using external validation datasets including 2276 pathology images. This study opens up fields of machine learning analysis for discovering uncharted knowledge.

Deep learning algorithms have achieved great success in cancer image classification. However, it is imperative to understand the differences between the deep learning and human approaches. Using an explainable model, we aimed to compare the deep learning-focused regions of magnetic resonance (MR) images with cancerous locations identified by radiologists and pathologists. First, 307 prostate MR images were classified using a well-established deep neural network without locational information of cancers. Subsequently, we assessed whether the deep learning-focused regions overlapped the radiologist-identified targets. Furthermore, pathologists provided histopathological diagnoses on 896 pathological images, and we compared the deep learning-focused regions with the genuine cancer locations through 3D reconstruction of pathological images. The area under the curve (AUC) for MR images classification was sufficiently high (AUC = 0.90, 95% confidence interval 0.87–0.94). Deep learning-focused regions overlapped radiologist-identified targets by 70.5% and pathologist-identified cancer locations by 72.1%. Lymphocyte aggregation and dilated prostatic ducts were observed in non-cancerous regions focused by deep learning. Deep learning algorithms can achieve highly accurate image classification without necessarily identifying radiological targets or cancer locations. Deep learning may find clues that can help a clinical diagnosis even if the cancer is not visible.

Accurate prostate cancer screening is imperative for reducing the risk of cancer death. Ultrasound imaging, although easy, tends to have low resolution and high inter-observer variability. Here, we show that our integrated machine learning approach enabled the detection of pathological high-grade cancer by the ultrasound procedure. Our study included 772 consecutive patients and 2899 prostate ultrasound images obtained at the Nippon Medical School Hospital. We applied machine learning analyses using ultrasound imaging data and clinical data to detect high-grade prostate cancer. The area under the curve (AUC) using clinical data was 0.691. On the other hand, the AUC when using clinical data and ultrasound imaging data was 0.835 (p = 0.007). Our data-driven ultrasound approach offers an efficient tool to triage patients with high-grade prostate cancers and expands the possibility of ultrasound imaging for the prostate cancer detection pathway.

Prostate cancer (PCa) is a clinically heterogeneous disease. Predicting clinically significant PCa with low–intermediate prostate-specific antigen (PSA), which often includes aggressive cancers, is imperative. This study evaluated the predictive accuracy of deep learning analysis using multimodal medical data focused on clinically significant PCa in patients with PSA ≤ 20 ng/mL. Our cohort study included 178 consecutive patients who underwent ultrasound-guided prostate biopsy. Deep learning analyses were applied to predict clinically significant PCa. We generated receiver operating characteristic curves and calculated the corresponding area under the curve (AUC) to assess the prediction. The AUC of the integrated medical data using our multimodal deep learning approach was 0.878 (95% confidence interval [CI]: 0.772–0.984) in all patients without PSA restriction. Despite the reduced predictive ability of PSA when restricted to PSA ≤ 20 ng/mL (n = 122), the AUC was 0.862 (95% CI: 0.723–1.000), complemented by imaging data. In addition, we assessed clinical presentations and images belonging to representative false-negative and false-positive cases. Our multimodal deep learning approach assists physicians in determining treatment strategies by predicting clinically significant PCa in patients with PSA ≤ 20 ng/mL before biopsy, contributing to personalized medical workflows for PCa management.

Peritoneal washing cytology (CY) in patients with pancreatic cancer is mainly used for staging; however, it may also be used to evaluate the intraperitoneal status to predict a more accurate prognosis. Here, we investigated the potential of deep learning of CY specimen images for predicting the 1-year prognosis of pancreatic cancer in CY-positive patients. CY specimens from 88 patients with prognostic information were retrospectively analyzed. CY specimens scanned by the whole slide imaging device were segmented and subjected to deep learning with a Vision Transformer (ViT) and a Convolutional Neural Network (CNN). The results indicated that ViT and CNN predicted the 1-year prognosis from scanned images with accuracies of 0.8056 and 0.8009 in the area under the curve of the receiver operating characteristic curves, respectively. Patients predicted to survive 1 year or more by ViT showed significantly longer survivals by Kaplan-Meier analyses. The cell nuclei found to have a negative prognostic impact by ViT appeared to be neutrophils. Our results indicate that AI-mediated analysis of CY specimens can successfully predict the 1-year prognosis of patients with pancreatic cancer positive for CY. Intraperitoneal neutrophils may be a novel prognostic marker and therapeutic target for CY-positive patients with pancreatic cancer.

Machine learning has shown promise in medical image classification. However, its generalizability remains challenging. Here, we show that data-efficient pre-surgical prognostication of prostate cancer from biopsy specimens is enabled by versatile feature extraction from whole-mount histopathology and a clinically informed intermediate reasoning step. With data from multiple institutions, our pipeline resolved dual-domain shifts across specimen types and institutions and achieved consistent external validation, reinforced by comprehensive analyses of generalizability. This highlights the robustness of our prognostic approach when compared to the Gleason grading system. We establish an equitable, interpretable, and clinically applicable framework, supporting actionable decisions for prognosis and treatment planning, even in limited real-world clinical environments.