The Image Biomarker Standardization Initiative validated consensus-based reference values for 169 radiomics features, thus enabling calibration and verification of radiomics software. Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpos...

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Background Radiomics extracts features from medical images more precisely and more accurately than visual assessment. However, radiomics features are affected by CT scanner parameters such as reconstruction kernel or section thickness, thus obscuring underlying biologically important texture features. Purpose To investigate whether a compensation method could correct for the variations of radiomic feature values caused by using different CT protocols. Materials and Methods Phantom data involving 10 texture patterns and 74 patients in cohorts 1 (19 men; 42 patients; mean age, 60.4 years; September-October 2013) and 2 (16 men; 32 patients; mean age, 62.1 years; January-September 2007) scanned by using different CT protocols were retrospectively included. For any radiomic feature, the compensation approach identified a protocol-specific transformation to express all data in a common space that were devoid of protocol effects. The differences in statistical distributions between protocols were assessed by using Friedman tests before and after compensation. Principal component analyses were performed on the phantom data to evaluate the ability to distinguish between texture patterns after compensation. Results In the phantom data, the statistical distributions of features were different between protocols for all radiomic features and texture patterns (P < .05). After compensation, the protocol effect was no longer detectable (P > .05). Principal component analysis demonstrated that each texture pattern was no longer displayed as different clusters corresponding to different imaging protocols, unlike what was observed before compensation. The correction for scanner effect was confirmed in patient data with 100% (10 of 10 features for cohort 1) and 98% (87 of 89 features for cohort 2) of P values less than .05 before compensation, compared with 30% (three of 10) and 15% (13 of 89) after compensation. Conclusion Image compensation successfully realigned feature distributions computed from different CT imaging protocols and should facilitate multicenter radiomic studies. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Steiger and Sood in this issue.

The impact of PET image acquisition and reconstruction parameters on SUV measurements or radiomic feature values is widely documented. This scanner effect is detrimental to the design and validation of predictive or prognostic models and limits the use of large multicenter cohorts. To reduce the impact of this scanner effect, the ComBat method has been proposed and is now used in various contexts. The purpose of this article is to explain and illustrate the use of ComBat based on practical examples. We also give examples in which the ComBat assumptions are not met and, thus, in which ComBat should not be used.

Introduction: Several reports have shown that radiomic feature values are affected by the acquisition and reconstruction parameters, thus hampering multicenter studies. We propose a method to standardize features measured from Positron Emission Tomography (PET) images obtained using different imaging protocols to remove the center effect while preserving patient-specific effects. Methods: Pre-treatment ¹⁸F-FDG-PET images of patients with breast cancer were included. In Department A, 63 patients were scanned using a Gemini Time-Of-Flight-PET/Computed-Tomography scanner including 16 triple-negative lesions (TN). In Department B, 74 patients underwent a PET on a GE Discovery 690 Scanner including 15 TN lesions. PET images from Department A were also smoothed using a Gaussian filter to mimic data from a third Department called Department A-S. The primary tumor was segmented to get a tumor volume of interest (VOI) and a spherical VOI was also set in healthy liver tissue. Three Standardized Uptake Values (SUVs) and 6 textural features were computed in all VOI using LIFEx software. A harmonization method, ComBat, initially described for genomic data, was used to estimate the department effect based on the observed feature values. Feature distributions in each department were compared before and after harmonization. Results: In healthy liver tissue, the feature distributions were significantly different for 4 out of 9 features between Departments A and B, and for 6 out of 9 between Departments A and A-S (p0.1). The same trend was observed in tumors with a realignment of feature values between the departments after ComBat. Identification of TN lesions was largely enhanced after harmonization when the cut-off values were determined on data from one department and applied to data from the other department. Conclusion: ComBat is efficient at removing the multicenter effect for textural features and SUVs. The method is easy to use, retains biological variations not related to a center effect, and does not require any feature recalculation. Such a harmonization allows for multicenter studies, external validation of radiomic models or cut-off values, and should facilitate the use of radiomic models in clinical practice.

Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. Key points: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics). Graphical