Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. Key points: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics). Graphical

Objectives While magnetic resonance imaging (MRI) is considered a helpful diagnostic tool in breast imaging, discussions are ongoing about appropriate protocols and indications. The European Society of Breast Imaging (EUSOBI) launched a survey to evaluate the utilisation of breast MRI in clinical practice. Methods An online survey reviewed by the EUSOBI board and committees was distributed amongst members. The questions encompassed: training and experience; annual breast MRI and MRI-guided-intervention workload; examination protocols; indications; reporting habits and preferences. Data were summarised and subgroups compared using χ² test. Results Of 647 EUSOBI members, 177 (27.4%) answered the survey. The majority were radiologists (90.5%), half of them based in academic centres (51.9%). Common indications for MRI included cancer staging, treatment monitoring, high-risk screening and problem-solving, and differed significantly between countries (p≤0.03). Structured reporting and BI-RADS were mostly used. Breast radiologists with ≤10 years of experience preferred inclusion of additional techniques, such as T2/STIR (p=0.03) and DWI (p=0.08) in the scan protocol. MRI-guided interventions were performed by a minority of participants (35.4%). Conclusions The utilisation of breast MRI in clinical practice is generally in line with international recommendations. There are substantial differences between countries. MRI-guided interventions and functional MRI parameters are not widely available. Key points • MRI is commonly used for the detection and characterisation of breast lesions. • Clinical practice standards are generally in line with current recommendations. • Standardised criteria and diagnostic categories (mainly BI-RADS) are widely adopted. • Younger radiologists value additional techniques, such as T2/STIR and DWI. • MRI-guided breast biopsy is not widely available.

Radiomics is a method to extract detailed information from diagnostic images that cannot be perceived by the naked eye. Although radiomics research carries great potential to improve clinical decision-making, its inherent methodological complexities make it difficult to comprehend every step of the analysis, often causing reproducibility and generalizability issues that hinder clinical adoption. Critical steps in the radiomics analysis and model development pipeline—such as image, application of image filters, and selection of feature extraction parameters—can greatly affect the values of radiomic features. Moreover, common errors in data partitioning, model comparison, fine-tuning, assessment, and calibration can reduce reproducibility and impede clinical translation. Clinical adoption of radiomics also requires a deep understanding of model explainability and the development of intuitive interpretations of radiomic features. To address these challenges, it is essential for radiomics model developers and clinicians to be well-versed in current best practices. Proper knowledge and application of these practices is crucial for accurate radiomics feature extraction, robust model development, and thorough assessment, ultimately increasing reproducibility, generalizability, and the likelihood of successful clinical translation. In this article, we have provided researchers with our recommendations along with practical examples to facilitate good research practices in radiomics. Key Points: Radiomics’ inherent methodological complexity should be understood to ensure rigorous radiomic model development to improve clinical decision-making. Adherence to radiomics-specific checklists and quality assessment tools ensures methodological rigor. Use of standardized radiomics tools and best practices enhances clinical translation of radiomics models.

This article provides radiologists with practical recommendations for evaluating AI performance in radiology, ensuring alignment with clinical goals and patient safety. It outlines key performance metrics, including overlap metrics for segmentation, test-based metrics (e.g., sensitivity, specificity, and area under the receiver operating characteristic curve), and outcome-based metrics (e.g., precision, negative predictive value, F1-score, Matthews correlation coefficient, and area under the precision-recall curve). Key recommendations emphasize local validation using independent datasets, selecting task-specific metrics, and considering deployment context to ensure real-world performance matches claimed efficacy. Common pitfalls, such as overreliance on a single metric, misinterpretation in low-prevalence settings, and failure to account for clinical workflow, are addressed with mitigation strategies. Additional guidance is provided on threshold selection, prevalence-adjusted evaluation, and AI-generated image quality assessment. This guide equips radiologists to critically evaluate both commercially available and in-house developed AI tools, ensuring their safe and effective integration into clinical practice. Clinical relevance statement: This review provides guidance on selecting and interpreting AI performance metrics in radiology, ensuring clinically meaningful evaluation and safe deployment of AI tools. By addressing common pitfalls and promoting standardized reporting, it supports radiologists in making informed decisions, ultimately improving diagnostic accuracy and patient outcomes. Key Points: Radiologists must evaluate performance metrics as they reflect acceptable performance in specific datasets but do not guarantee clinical utility. Independent evaluation tailored to the clinical setting is essential . Performance metrics must align with the intended task of the AI application—segmentation, detection, or classification—and be selected based on domain knowledge and clinical context . Sensitivity, specificity, area under the ROC curve, and accuracy must be interpreted with prevalence-dependent metrics (e.g., precision, F1 score, and Matthew’s correlation coefficient) calculated for the target population to ensure safe and effective clinical use .

Background Breast MRI is the most sensitive method for the detection of breast cancer and is an integral part of modern breast imaging. On the other hand, interpretation of breast MRI exams is considered challenging due to the complexity of the available information. Clinical decision rules that combine diagnostic criteria in an algorithm can help the radiologist to read breast MRI by supporting objective and largely experience-independent diagnosis. Method Narrative review. In this article, the Kaiser Score (KS) as a clinical decision rule for breast MRI is introduced, its diagnostic criteria are defined, and strategies for clinical decision making using the KS are explained and discussed. Results The KS is based on machine learning and has been independently validated by international research. It is largely independent of the examination technique that is used. It allows objective differentiation between benign and malignant contrast-enhancing breast MRI findings using diagnostic BI-RADS criteria taken from T2w and dynamic contrast-enhanced T1w images. A flowchart guides the reader in up to three steps to determine a score corresponding to the probability of malignancy that can be used to assign a BI-RADS category. Individual decision making takes the clinical context into account and is illustrated by typical scenarios. Key Points: Citation Format