Recent technological advances in healthcare have led to unprecedented growth in patient data quantity and diversity. While artificial intelligence (AI) models have shown promising results in analyzing individual data modalities, there is increasing recognition that models integrating multiple complementary data sources, so-called multimodal AI, could enhance clinical decision-making. This scoping review examines the landscape of deep learning-based multimodal AI applications across the medical domain, analyzing 432 papers published between 2018 and 2024. We provide an extensive overview of multimodal AI development across different medical disciplines, examining various architectural approaches, fusion strategies, and common application areas. Our analysis reveals that multimodal AI models consistently outperform their unimodal counterparts, with an average improvement of 6.2 percentage points in AUC. However, several challenges persist, including cross-departmental coordination, heterogeneous data characteristics, and incomplete datasets. We critically assess the technical and practical challenges in developing multimodal AI systems and discuss potential strategies for their clinical implementation, including a brief overview of commercially available multimodal AI models for clinical decision-making. Additionally, we identify key factors driving multimodal AI development and propose recommendations to accelerate the field's maturation. This review provides researchers and clinicians with a thorough understanding of the current state, challenges, and future directions of multimodal AI in medicine.

The purpose of this paper is to develop and validate a delta-radiomics score to predict the response of individual colorectal cancer liver metastases (lmCRC) to first-line FOLFOX chemotherapy. Three hundred one lmCRC were manually segmented on both CT performed at baseline and after the first cycle of first-line FOLFOX, and 107 radiomics features were computed by subtracting textural features of CT at baseline from those at timepoint 1 (TP1). LmCRC were classified as nonresponders (R−) if they showed progression of disease (PD), according to RECIST1.1, before 8 months, and as responders (R+), otherwise. After feature selection, we developed a decision tree statistical model trained using all lmCRC coming from one hospital. The final output was a delta-radiomics signature subsequently validated on an external dataset. Sensitivity, specificity, positive (PPV), and negative (NPV) predictive values in correctly classifying individual lesions were assessed on both datasets. Per-lesion sensitivity, specificity, PPV, and NPV were 99%, 94%, 95%, 99%, 85%, 92%, 90%, and 87%, respectively, in the training and validation datasets. The delta-radiomics signature was able to reliably predict R− lmCRC, which were wrongly classified by lesion RECIST as R+ at TP1, (93%, averaging training and validation set, versus 67% of RECIST). The delta-radiomics signature developed in this study can reliably predict the response of individual lmCRC to oxaliplatin-based chemotherapy. Lesions forecasted as poor or nonresponders by the signature could be further investigated, potentially paving the way to lesion-specific therapies.

In the last years, several studies demonstrated that low-aggressive (Grade Group (GG) ≤ 2) and high-aggressive (GG ≥ 3) prostate cancers (PCas) have different prognoses and mortality. Therefore, the aim of this study was to develop and externally validate a radiomic model to noninvasively classify low-aggressive and high-aggressive PCas based on biparametric magnetic resonance imaging (bpMRI). To this end, 283 patients were retrospectively enrolled from four centers. Features were extracted from apparent diffusion coefficient (ADC) maps and T2-weighted (T2w) sequences. A cross-validation (CV) strategy was adopted to assess the robustness of several classifiers using two out of the four centers. Then, the best classifier was externally validated using the other two centers. An explanation for the final radiomics signature was provided through Shapley additive explanation (SHAP) values and partial dependence plots (PDP). The best combination was a naïve Bayes classifier trained with ten features that reached promising results, i.e., an area under the receiver operating characteristic (ROC) curve (AUC) of 0.75 and 0.73 in the construction and external validation set, respectively. The findings of our work suggest that our radiomics model could help distinguish between low- and high-aggressive PCa. This noninvasive approach, if further validated and integrated into a clinical decision support system able to automatically detect PCa, could help clinicians managing men with suspicion of PCa.

Colorectal cancer (CRC) has the second-highest tumor incidence and is a leading cause of death by cancer. Nearly 20% of patients with CRC will have metastases (mts) at the time of diagnosis, and more than 50% of patients with CRC develop metastases during their disease. Unfortunately, only 45% of patients after a chemotherapy will respond to treatment. The aim of this study is to develop and validate a machine learning algorithm to predict response of individual liver mts, using CT scans. Understanding which mts will respond or not will help clinicians in providing a more efficient per-lesion treatment based on patient specific response and not only following a standard treatment. A group of 92 patients was enrolled from two Italian institutions. CT scans were collected, and the portal venous phase was manually segmented by an expert radiologist. Then, 75 radiomics features were extracted both from 7x7 ROIs that moved across the image and from the whole 3D mts. Feature selection was performed using a genetic algorithm. Results are presented as a comparison of the two different approaches of features extraction and different classification algorithms. Accuracy (ACC), sensitivity (SE), specificity (SP), negative and positive predictive values (NPV and PPV) were evaluated for all lesions (per-lesion analysis) and patients (per-patient analysis) in the construction and validation sets. Best results were obtained in the per-lesion analysis from the 3D approach using a Support Vector Machine as classifier. We reached on the training set an ACC of 81%, while on test set, we obtained SE of 76%, SP of 67%, PPV of 69% and NPV of 75%. On the validation set a SE of 61%, SP of 60%, PPV of 57% and NPV of 64% were reached. The promising results obtained in the validation dataset should be extended to a larger cohort of patient to further validate our method.Clinical Relevance— to develop a radiomics signatures predicting single liver mts response to therapy. A personalized mts approach is important to avoid unnecessary toxicity offering more suitable treatments and a better quality of life to oncological patients.

In conclusion, there is an absolute need for AI regulations in the field of medicine in general, both in clinical and scientific practice. The role of the AI trifecta-as defined as the alliance between developers, doctors, and patients-would be to define the spaces, times, and ways of AI in medicine. In this context, the branch of Urology has the chance to lead the way, perpetuating its propensity for innovation, technology, and research.

Automatic segmentation of the prostate on Magnetic Resonance Imaging (MRI) is one of the topics on which research has focused in recent years as it is a fundamental first step in the building process of a Computer aided diagnosis (CAD) system for cancer detection. Unfortunately, MRI acquired in different centers with different scanners leads to images with different characteristics. In this work, we propose an automatic algorithm for prostate segmentation, based on a U-Net applying transfer learning method in a bi-center setting. First, T2w images with and without endorectal coil from 80 patients acquired at Center A were used as training set and internal validation set. Then, T2w images without endorectal coil from 20 patients acquired at Center B were used as external validation. The reference standard for this study was manual segmentation of the prostate gland performed by an expert operator. The results showed a Dice similarity coefficient >85% in both internal and external validation datasets.Clinical Relevance— This segmentation algorithm could be integrated into a CAD system to optimize computational effort in prostate cancer detection.